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Preface 


The  purpose  of  this  study  was  to  examine  software 
maintenance  costs  and  its  effect  on  Air  Force  life  cycle 
costs.  Only  recently  has  work  been  devoted  to  the  study 
of  methods  for  maintaining  software  once  it  has  been 
developed.  Maintenance  costs  are  rising  within  industry 
and  the  Air  Force  at  an  alarming  rate.  If  efforts  are  not 
taken  to  reduce  the  current  rise  in  maintenance,  time  and 
resources  will  not  be  available  to  develop  new  software. 

I  would  like  to  thank  Professor  James  D.  Meadows, 
iny  advisor,  for  his  support,  advice,  and  time.  Captain 
Roger  Davis  has  my  appreciation  for  providing  many  hours 
of  help  by  reading  my  thesis  and  giving  constructive  com¬ 
ments.  I  would  also  like  to  thank  Mr.  Fred  Wixon  of 
HQ  SISC/SCD  at  Gunter  AFS  AL  for  his  help  with  the  AFR 
700-19  database.  I  greatly  appreciate  the  help  and  sup¬ 
port  of  my  family  during  this  thesis  effort.  Lastly,  I 
wish  to  give  special  thanks  to  my  wife,  Barbara,  for  her 
innumerable  hours  of  support  on  my  behalf.  She  richly 
deserves  more  appreciation  than  is  possible  in  words. 

—  Robert  E.  NeSmith  II 
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Software  maintenance  is  a  growing  concern  through¬ 
out  the  software  community.  Due  to  the  rising  cost  of 
computer  sofcware  and  the  even  greater  increase  in  the 
software  maintenance  share  of  the  budget,  maintenance  is 
becoming  the  major  cost  in  a  data  processing  organization, 
This  thesis  examines  the  maintenance  costs  of 

J-'' 

Air  Force  "organic11  software  for  the  last  thirty  years. 
Generally,  the  cost  per  line  of  code  and  the  total  costs 
are  risAng  for  the  large  scale  automatic  data  processing 
computer  systems.  Contractor  developed  software  is  also 

examined  and  its  influence  on  Air  Force  software  costs. 
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A  STUDY  OF  SOFTWARE  MAINTENANCE  COSTS  OF  AIR  FORCE 


LARGE  SCALE  COMPUTER  SYSTEMS 

1 .  Overview 

Introduction 

Industry  cost  studies  of  software  have  shown  that 
of  the  various  life  cycle  stages  of  a  software  system, 
software  maintenance  is  the  most  expensive  portion  (25:1). 


Maintenance  has  been  defined  as  the  performance  of  those 
activities  required  to  keep  a  software  system  operational 
and  responsive  to  its  users  after  it  has  been  accepted  and 
placed  into  production.  Maintenance  has  also  been  found 
to  consume  over  50  percent  of  the  software  lire  cycle 
budget  (13:65;  19:<»;  14:4).  Studies  performed  in  the 
1970s  and  1980s  have  shown  that  the  longer  an  error 
remains  undetected,  the  more  it  will  cost  to  correct  that 
error  (15:116).  For  example,  an  error  found  in  the  design 
stage  is  cheaper  to  correct  than  an  error  found  in  a  later 
stage  such  as  testing.  Therefore,  emphasis  on  software 
maintainability  early  in  the  development  process  can  reduce 
costs  by  finding  and  correcting  any  of  the  various  design 
or  program  errors  that  can  occur  before  a  system  is  opera¬ 
tional  (25:2).  Industry  estimates  suggest  that  once  a 
system  is  operational,  program  changes  cost  up  to 


thirty-seven  times  more  than  program  changes  made  during 
development  (9:51). 

Further,  we  can  expect  software  costs  to  continue 
to  increase. 

The  annual  cost  of  computer  software  in  the  United 
States  in  1980  was  approximately  40  billion  dollars  or 
about  2  percent  of  the  Gross  National  Product  (GNP)  . 

Its  rate  of  growth  is  considerably  greater  than  that 
of  the  economy  in  general.  (5:17) 

The  combined  cost  for  software  development  and  maintenance 
is  predicted  to  be  13  percent  of  the  GNP  by  1990.  The 
ratio  of  computer  software  costs  to  overall  computer  sys¬ 
tem  costs  has  also  grown  tremendously.  In  the  1950s,  90 
percent  of  the  cost  of  a  computer  system  was  due  to  the 
cost  of  computer  hardware.  Currently,  more  than  8G  percent 
of  the  system  cost  is  attributed  to  the  cost  of  software 
(5:18;  25:1).  See  Figure  1-1. 
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Fig.  1-1.  Hardware/Software  Cost  Trends  (5:18) 


Another  factor  which  will  cause  software  costs  to 
rise  is  the  increasing  reliance  on  software  to  accomplish 
tasks  previously  performed  by  hardware.  "As  software  per¬ 
meates  virtually  every  defense  system,  cost  effective  and 
reliable  software  becomes  increasingly  urgent"  (3:52). 

The  Department  of  Defense  (DoD)  estimated  in  1982  that 
DoD's  costs  for  new  software  will  increase  by  a  factor  of 
six  to  $32  billion  by  1990  (3:54). 

Industry  tracks  all  efforts  according  to  the  costs 
incurred.  Eventually  all  costs  incurred  in  the  operation 
and  maintenance  of  software  is  charged  to  either  the  user 
or  the  developer  depending  upon  the  situation.  If  the 
software  operates  properly  and  produces  the  desired  output, 
then  the  operation  expense  is  charged  to  the  user.  If 
the  software  runs  improperly,  then  the  software  goes  back 
to  the  development  office  for  repair  and  the  costs  are 
incurred  by  the  developer. 

However,  it  is  difficult  to  determine  exact  soft- 

I 

ware  costs  in  the  Air  Force  (AF) .  Since  the  Air  Force 

i 

usually  develops  software  within  the  same  unit  or  command, 
the  costs  all  come  out  of  the  same  budget.  Historically, 

I 

|  only  efforts  involving  initial  development  have  been 

i 

|  tracked.  These  efforts  included  the  number  of  lines  of 

i 

j  code  written,  the  number  of  programmer  hours  expended,  the 

i 

number  of  months  until  delivery,  etc.  The  tracking  of 
maintenance  costs  was  not  considered  a  part  of  the  software 

1 

1 
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costs.  In  the  early  1980s,  DoD  instituted  steps  to  facili¬ 
tate  better  tracking  of  software  costs.  This  was  due  pri¬ 
marily  to  the  increased  emphasis  of  "Fraud,  Waste  and 
Abuse"  and  reduced  spending  levels. 

Finally,  new  and  existing  software  systems  are 
developed  and  maintained  by  the  limited  number  of  program¬ 
mers  that  exist  in  industry  and  government.  At  the  present 
rate  of  growth  in  maintenance  requirements  for  existing 
and  new  systems,  it  is  expected  that  all  programmers  will 
do  maintenance  work  without  additional  time  for  developing 
new  software  after  1990  (10:74;  28:61). 

These  factors  combined  truly  present  a  challenge 
to  the  managers  of  Air  Force  software  of  today  and  the  near 
future.  The  purpose  of  this  thesis  is  to  analyze  main¬ 
tenance  costs  of  Air  Force  software  and  discuss  concepts 
which  affect  these  costs. 

Problem  Statement 

Due  to  the  emphasis  on  budget  reductions,  ways  to 
reduce  costs  become  increasingly  important.  Industry 
studies  show  that  software  maintenance  costs  are  requiring 
a  greater  share  of  the  overall  software  budget.  The  Air 
Force  has  taken  an  active  role  in  controlling  software 
development  costs  but  has  not  adequately  assessed  the  costs 
of  continual  maintenance  on  its  library  of  software  used  in 
data  processing  centers.  Several  studies  have  been 


performed  on  the  costs  of  "embedded"  software  for  weapon 
systems.  Few  comparable  studies  have  been  completed  on 
Air  Force  data  processing  or  information  systems  software 
(20:7) .  An  understanding  of  the  composition  of  these  costs 
is  necessary  before  steps  to  reduce  them  can  take  place. 

Justification 

A  literature  search  found  maintenance  studies  per¬ 
formed  on  industry-developed  software  and  on  Air  Force 
embedded  software  but  not  on  Air  Force  software  for  large 
automated  data  processing  (ADP)  machines.  Air  Force  soft¬ 
ware  maintenance  presents  one  area  where  significant  cost 
reductions  can  be  achieved  to  help  meet  reduced  funding 
levels  imposed  by  Congress.  Methods  used  successfully  by 
industry  to  reduce  software  maintenance  costs  may  provide 
ways  for  the  Air  Force  to  reduce  its  software  maintenance 
costs.  The  information  derived  from  this  research  will  aid 
Air  Force  managers  in  planning  and  assigning  resources  to 
the  maintenance  of  Air  Force  software. 


Objectives  of  Research 

This  thesis  builds  on  a  thesis  prepared  by 
Major  Robert  E.  Childress,  an  AFIT  1985  graduate.  His 
thesis  is  entitled  Contractor  versus  Organic  Maintenance 
for  Space  Command  Automatic  Data  Processing  Equipment. 
Although  Major  Childress  looked  primarily  at  manpower 
shortages,  this  thesis  looks  at  costs.  The  overall 


objective  of  this  thesis  is  to  analyze  historical  software 
costs  by  examining  the  effect  of  the  programming  language 
used,  the  number  of  lines  of  code,  the  magnitude  of  main¬ 
tenance  and  development  costs,  year,  contractor,  and  major 
command  user.  This  information  will  allow  evaluation  of 
the  following: 

1.  What  has  been  the  change  in  software  develop¬ 
ment  costs  over  time? 

2.  Have  software  development  costs  shown  the  same 
increase  independent  of  the  AF  command  or  the  programming 
language  used? 

3.  What  have  been  the  costs  of  software  main¬ 
tenance  for  large  data  processing  systems? 

4 .  Have  software  maintenance  costs  shown  the  same 
increase  independent  of  the  AF  command  or  the  programming 
language  used? 

5.  What  differences  are  found  between  software 
development  and  maintenance  costs? 

Scope 

Research  of  projected  software  requirements  indi¬ 
cates  that  software  needs  are  increasing  while  at  the  same 
time,  budget  constraints  are  being  imposed  upon  DoD  by 
Congress.  Therefore,  this  thesis  effort  has  potential 
for  application  in  all  Air  Force  and  possibly  all  DoD 
software  efforts.  The  information  provided  in  this  thesis 
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will  provide  the  reader  with  an  awareness  of  the  main¬ 
tenance  costs  within  the  Air  Force  in  terms  of  the  com¬ 
puter  language  used  and  the  user  command.  It  will  also 
provide  current  techniques,  methodology,  and  direction  that 
can  reduce  maintenance  costs  for  current  and  future  soft¬ 
ware  systems.  With  the  limited  AF  resources  available  for 
developing  and  maintaining  software,  efforts  on  the  part  of 
the  user  command  can  reduce  these  growing  costs  and  make 
better  use  of  limited  resources. 

The  major  constraint  for  this  project  is  the  amount 
of  time  allowed  by  AFIT  for  thesis  research.  The  period 
allows  fifteen  months  of  research  work.  Another  constraint 
concerns  the  accounting  methods  used  by  Air  Force  organiza¬ 
tions  in  determining  software  development  and  maintenance 
costs.  Also,  this  study  does  not  examine  the  subject  of 
dedicated,  system  resident,  or  embedded  software  costs. 
Instead,  only  those  software  systems  running  on  automated 
or  large-scale  data  processing  equipment  by  the  Air  Force 
and  DoD  are  examined.  Studies  have  been  previously  per¬ 
formed  on  the  software  used  in  embedded  systems.  These 
studies  show  even  higher  maintenance  costs  than  on  large 
ADP  systems  mentioned  in  this  thesis.  Maintenance  on  large 
ADP  systems  can  be  greater  than  50  percent  of  the  total 
life  cycle  costs  or  up  to  $2000  per  line  of  code,  whereas, 
embedded  software  maintenance  costs  can  run  as  high  as 
$4000  per  line  of  code  <26;  16:16). 
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II.  Background 


The  U.S.  Government  is  the  largest  user  of  com¬ 
puters  in  the  world,  with  over  17,000  computers  running 
over  5  million  software  packages.  Most  of  these  packages 
were  developed  at  the  cost  of  hundreds  of  staff  years 
amounting  to  billions  of  dollars  (4:115). 

This  chapter  is  organized  into  four  basic  sections. 
The  first  section  gives  background  on  software  development 
and  maintenance.  Section  two  describes  software  life  cycle 
costs.  Section  three  presents  some  areas  that  increase  the 
cost  of  maintenance.  Finally,  section  four  presents  ways 
to  reduce  the  costs  of  software  maintenance. 

Section  1 

Software  Development.  As  software  development 
costs  have  risen,  so  have  the  associated  software  mainte¬ 
nance  costs.  In  the  1980s,  the  U.S.  Government  is  spending 
on  the  average  of  $900  million  annually  to  support  software. 
It  is  predicted  that  this  will  rise  more  than  ten-fold  by 
the  1990s  (4:115).  Programmers  today  spend  more  time 
adapting  and  correcting  existing  software  than  writing 
new  software  (4:115) .  This  means  that  users  spend  more 
on  maintenance  of  software  than  on  development  of  new  code. 

Since  software  development  is  time  intensive, 
economical  and  efficient  techniques  to  maintain  existing 
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software  must  be  used  to  reduce  costs  and  conserve 
resources.  Ideally,  these  techniques  should  begin  when 
the  program  is  developed  (25:5). 

During  the  past  thirty  years  the  number  of  software 
programs  has  grown  at  an  exponential  rate.  Many  of  these 
software  programs  are  still  in  use  today.  This  is  espe¬ 
cially  true  of  the  government's  computer  systems.  A  recent 
study  states  that  the  average  age  of  government  software  is 
ten  years,  three  months  (30:1).  This  is  almost  three  times 
the  age  of  software  in  industry  (22:2).  Each  of  these 
programs  added  to  a  computer  system's  library  increases 
the  operations  and  maintenance  portion  of  the  total  com¬ 
puter  facilities  budget  (25:5).  This  leaves  less  funds 
available  for  the  development  requirements  (28:61) . 

Software  Maintenance.  Maintenance  covers  three 
areas:  Perfective,  Corrective,  and  Adaptive.  Definitions 
of  these  terms  are  found  in  Appendix  A.  Perfective 
requires  the  most  time  from  the  maintenance  programmer.  A 
study  by  the  National  Bureau  of  Standards  found  the  main¬ 
tenance  programmer  spent  55  percent  of  his  time  perfecting 
software,  25  percent  adapting,  leaving  only  20  percent  for 
correction  of  errors  (25:1). 

Any  maintenance  work  performed  on  a  system  tends 
to  disrupt  the  complete  program  structure.  Therefore,  it 
has  been  found  that  the  longer  the  software  has  been  used. 
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the  greater  the  chance  of  usage  problems  due  to  the  main¬ 
tenance  completed  on  the  software  (11:102).  Enhancements 
will  be  implemented,  hardware  will  be  upgraded,  and 
untested  "bugs"  will  pop  up.  All  will  require  programmer 
work  to  run  properly.  Additionally,  much  of  the  older  soft¬ 
ware  is  custom-made,  using  techniques  and  languages  that 
complicate  rapid  problem  analysis  and  repair  (28:61). 

In  many  cases  the  original  writers  of  software  have 
left  the  organization,  leaving  maintenance  to  personnel 
unfamiliar  with  the  software.  Documentation  describing 
the  function  of  the  program  is  often  out  of  date  or  non¬ 
existent,  decreasing  the  available  reference  material. 

This  in  turn  increases  the  work  load  on  software  mainte¬ 
nance  personnel.  Some  computer  software  maintenance  is 
needed  immediately  due  to  the  critical  nature  of  the  soft¬ 
ware.  The  maintenance  programmer  does  not  have  the  luxury 
of  months  to  study  the  program  and  make  any  trial  runs 
(33:22) . 

A  National  Bureau  of  Standards  study  examined  the 
tasks  involved  in  software  maintenance.  These  tasks  are 
not  always  performed  by  just  one  programmer  or  office,  but 
the  study  shows  the  reasons  why  maintenance  costs  can  add 
up.  The  breakdown  of  these  tasks  that  must  be  accomplished 
and  paid  for  is  listed  below  (19:12). 

1.  Requirements  Analysis 

2.  Design  Analysis 
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3.  User  Interface 

4 .  Design  Review 

5.  Problem  Report  Recording  and  Control 

6.  Configuration  Control 

7.  Database  Modification 

8.  Code  Modif  ication/Recanpilation 

9.  Code  Debugging /Module  Testing 

10.  Subsystem  Testing 

11.  System  Testing 

12.  Documentation  Modification 

13.  Standards  Audit 

14.  Code  Inspections/Walkthroughs 

15.  Test  Data  Generation 

16.  Management  Planning  and  Control 

17.  Field  Delivery 

18.  Software  Support  Development/Maintenance 

19.  Hardware  Support 

20.  Administrative  Support 

As  can  be  seen  from  these  tasks,  maintenance  is  basically 
a  microcosm  of  the  development  effort  (16:18)  . 

Despite  problems  with  software  and  its  use,  soft¬ 
ware  development  has  rapidly  expanded  and  software  main¬ 
tenance  has  begun  maturing  as  a  structured  science.  As 
computer  hardware  has  improved  in  cost,  size,  and  perform¬ 
ance,  software  capabilities  have  also  improved.  New 
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software  is  more  powerful,  easier  to  use,  and  more  readily 
available . 
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Software  maintenance  activities  are  affected  by 
software  development  and  operations.  Several  software- 
related  subjects  are  examined  that  will  provide  the  reader 
with  a  better  understanding  of  the  i  >le  of  maintenance  in 
the  total  software  picture. 

Section  II 

Software  Life  Cycle  Costs.  The  life  cycle  of  a 
computer  software  system  is  defined  as  the  period  from  its 
conception  until  it  is  no  longer  used  (15:29;  24:9).  The 
software  life  cycle  includes  six  basic  phases  (25:2): 

1.  Concept  and  Analysis 

2.  Requirement  Definition 

3 .  Design 

4 .  Code,  Debug  and  Test 

5.  Installation  and  Evaluation 

6.  Operation  and  Maintenance 

Software  life  cycle  costs  are  all  costs  incurred  in  these 
six  phases.  Often  software  costs  are  erroneously  thought 
to  occur  until  the  software  is  delivered  to  the  user.  As 
in  any  complex  product,  many  development  activities,  such 
as  documentation  or  structured  program  coding,  affect  the 
performance  of  software  as  well  as  the  maintenance  efforts 
required  during  the  software  life  cycle.  Any  activity 
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involved  in  the  development  or  maintenance  such  as  docu¬ 
mentation  updates  or  structured  coding  in  development  of 
the  software  will  affect  the  total  life  cycle  costs. 

Software  Cost  Estimation.  Prior  to  the  1970s, 
little  was  done  to  allow  the  proper  estimation  of  the  cost 
of  a  software  project.  Software  development  was  basically 
considered  more  of  a  nebulous  form  of  art  than  a  structured 
science.  This  meant  that  the  developer  did  not  have  a  firm 
idea  of  the  software  cost  until  after  the  project  had 
ended.  This  allowed  seme  projects  to  far  exceed  the  cost 
considered  to  be  the  dividing  point  between  cost-effective 
and  cost-excessive.  During  the  past  decade,  serious 
efforts  have  been  made  to  add  structure  and  controls  to 
software.  The  most  notable  concept  has  been  Top-Down 
Structured  Programming  (25:3,8). 

Cost  estimation  techniques  have  followed  the  struc¬ 
tured  development  approach.  Structured  cost  estimation 
techniques  generally  fall  into  three  categories:  Analogy, 
Bottom-up  and  Top-Down.  Each  technique  has  its  own 
strengths  and  limitations. 

Analogy  estimation  compares  programs  against  the 
cost  of  similar  programs.  The  cost  estimate  is  based  on 
actual  experience.  Often,  though,  there  is  no  "similar" 
program  to  compare  against  (15:12). 


Bottom-up  techniques  estimate  costs  by  estimating 
the  costs  of  exponents  or  units  of  the  system  and  then 
summing  these  costs.  Bottom-up  techniques  give  a  detailed 
basis  for  costs,  are  usually  accurate  and  stable,  and  can 
promote  individual  responsibility  for  the  software.  The 
problems  with  the  bottom-up  estimation  technique  are  that 
the  technique  does  not  capture  the  integration  costs,  is 
time  consuming,  and  requires  a  detailed  knowledge  of  the 
system  (15:12) . 

The  last  technique,  Top-Down,  is  also  called 
parametric.  Top-Down  estimates  software  system  costs 
based  on  design  parameters  which  can  be  partitioned  among 
components  or  life  cycle  phases.  It  is  fast  and  easy  to 
use  while  requiring  little  detailed  knowledge.  At  the 
same  time,  it  captures  the  system-level  costs.  The  weak¬ 
nesses  with  top-down  techniques  are  that  they  are  less 
stable  and  not  always  accurate  (15:12) .  Of  the  Top-Down 
models  developed,  one  of  the  most  noted  of  the  1980s  has 
been  the  Constructive  Cost  Model  or  "Cocomo"  developed  by 
Dr.  Barry  Boehm  of  TRW  (5:58).  Cocomo  is  one  of  the  non¬ 
proprietary  models  available.  Several  other  models,  each 
varying  in  the  number  of  software  factors  they  consider, 
are  also  in  use  today.  A  list  of  various  models,  for 
development  and  for  support,  is  shown  in  Tables  2-1  and 
2-2  *15:6,10). 
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All  of  the  models  have  a  common  thread.  Each 
requires  information  from  the  specific  organization  wishing 
to  estimate  future  software  costs.  No  one  formula  has  been 
derived  that  can  be  of  immediate  use  for  all  organizations 
(15:140-142).  Some  companies  have  the  know-how  and 
resources  to  produce  software  at  a  lower  "per  line"  or 
"per  project"  cost  than  others.  Therefore,  the  input  fac¬ 
tors  for  each  company  will  be  different. 

The  National  Bureau  of  Standards  reported  that 
"software  maintenance  can  account  for  up  to  seventy  percent 
of  each  software  dollar  allocated"  (13:65;  26).  The 
increase  in  maintenance  over  development  costs  is  two¬ 
fold.  Once  a  system  is  placed  into  production  its  main¬ 
tenance  costs  will  take  away  from  the  total  ADP  budget 
for  every  year  that  the  system  is  used  rather  than  just  a 
one-time  development  charge.  Secondly,  maintenance  has  a 
"ripple"  effect  on  other  lines  of  code  within  the  same 
software  system  (11:102) .  Every  line  changed  affects  other 
lines  of  code,  "snowballing"  the  changes  needed.  Once  the 
changes  have  been  made,  the  program  has  to  be  retested  and 
the  documentation  updated  (11:102;  18:4).  Over  time  the 
changes  cause  a  degradation  in  the  quality  and  structure  of 
the  software,  which  increases  the  rate  of  future  problems. 
All  of  these  changes  consume  maintenance  resources  (11:102). 


Section  III 


Maintenance  Programmers .  Software  development 
projects  are  usually  staffed  with  a  specific  mix  of  per¬ 
sonnel  skills  to  meet  the  requirements  of  the  application 
development.  People  with  different  skills  are  assigned  at 
the  various  phases  of  the  development  to  take  full  advan¬ 
tage  of  their  expertise.  In  contrast,  all  maintenance 
activities  for  a  particular  software  product  are  often 
performed  by  one  person,  acting  as  requirements  analyst, 
designer,  coder,  and  tester.  Although  this  is  not  always 
the  case,  the  separation  and  assembling  of  a  certain  mix 
of  skilled  individuals  is  more  typical  during  development 
than  during  maintenance  (24:11). 

Experienced  programmers  generally  prefer  to  develop 
new  systems  rather  than  work  on  software  maintenance. 
Therefore,  inexperienced  programmers  usually  are  assigned 
to  work  in  the  maintenance  shops  to  repair  the  defective 
programs.  This  places  the  responsibility  for  an  important 
piece  of  software  "on  the  shoulders"  of  someone  with  very 
little  knowledge  of  the  program  and  little  experience  in 
all  the  aspects  necessary  to  keep  it  running.  Often,  the 
new  programmer  either  is  promoted  out  of  the  shop  or 
becomes  so  dissatisfied  from  the  work  pressure  that  he 
quits.  This  places  the  burden  of  maintenance  on  someone 
else,  starting  the  cycle  all  over  again  (23:11). 


Most  software  systems  are  maintained  by  a  differ¬ 
ent  staff  of  programmers  than  those  who  developed  the 
original  code  (13:74;  24:1,11).  This  is  a  major  factor 
in  software  maintenance  where  the  software  is  on  average 
ten  years  old  (30:1)  .  People  move  on,  especially  in  the 
Air  Force.  The  usual  assignment  for  programmers  is  four 
years.  The  knowledge  that  took  years  to  develop  about  a 
system  usually  leaves  with  these  individuals.  The 
"corporate  knowledge"  in  the  organization  concerning  a 
software  package  can  be  a  tremendous  aid  to  the  mainte¬ 
nance  programmer  in  under standing  a  system  and  making  the 
necessary  changes  in  the  software.  Most  large  data  process¬ 
ing  organizations  have  their  development  personnel  in  a 
different  section  from  their  maintenance  personnel.  Indus¬ 
try  studies  have  shown  an  annual  turnover  of  28  percent  of 
the  personnel  in  a  data  processing  shop.  The  reduction  of 
a  stable  programming  staff  negatively  affects  the  corporate 
memory  of  the  organization.  The  increased  work  load  of 
unfamiliar  software  in  turn  can  affect  the  morale  of  the 
remaining  programmers  (2:807). 

All  of  this  can  lead  to  increased  downtime  on 
programs  needing  maintenance  (29:5).  One  way  to  relieve 
some  of  these  problems  is  to  provide  effective  programming 
tools  such  as  better  documentation  for  the  maintenance  pro¬ 
grammer.  Training  to  use  these  tools  is  especially  impor¬ 
tant  since  it  not  only  relieves  dissatisfaction  but  can 
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increase  the  programmer ' s  effectiveness  and  efficiency. 

This  can  be  critical  when  short  time  constraints  are 
involved  (2:808). 

Management  and  User  Influence.  A  National  Bureau 
of  Standards  study  identified  management  second  only  to 
costs  as  a  significant  area  of  concern  in  terms  of  software 
maintenance.  The  management  problems  involve: 

1.  Managing/recognizing  software  main*  nance  as 
a  separate  function 

2.  User  and  upper  level  management’s  perception 
of  software  maintenance 

3.  A  lack  of  goals,  standards,  and  criteria  to 
judge  performance  (metrics) 

4.  Managing  the  user  interface 

The  report  concluded  by  stating  that  the  significance  of 
software  maintenance  was  not  recognized  historically 
(24:13) . 

Management  and  the  user  exert  significant  influence 
in  software  maintenance.  Software  maintenance  can  be 
viewed  as  successive  iterations  of  the  development  phases, 
but  its  uniqueness  should  be  recognized  to  insure  effect- 
tive  management  (24:11).  First,  management  must  realize 
the  difficulty  of  maintenance  tasks.  Management  can  influ¬ 
ence  software  development  and  maintenance  since  it  manages 
the  motivation/ reward  system  of  the  organization  as  well  as 
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the  budget,  i.e.,  promotions  and  salary  raises.  The  indi¬ 
viduals  in  the  programming  offices  in  turn  will  plan  and 
develop  those  requirements  dictated  by  management  (27:49). 

Management  is  driven  by  user  wants  since  computer 
support  is  service  oriented.  If  the  user  understands  that 
structured  coding  and  documentation  will  aid  in  debugging 
software  problems  years  from  now,  then  the  user  can 
request  management  to  enforce  these  standards.  Management 
in  turn  can  develop  the  plans  needed  to  control  the  organi¬ 
zation  and  its  products  (2:807) .  Therefore,  training  users 
to  understand  the  "costs"  of  their  requests  in  terms  of 
time,  money,  personnel,  resources,  and  quality  of  product 
will  influence  the  type  of  requests  received  in  the  future 
(33:67)  . 


Air  Force  Issues.  Although  presented  from  an  indus 
try  viewpoint,  all  of  these  issues  reflect  current  situa¬ 
tions  in  the  Air  Force  that  affect  software  maintenance. 

In  some  cases,  the  problem  is  getting  worse  because  indus¬ 
try  programmer  salares  are  higher  than  Air  Force  programmer 
salaries.  Many  Air  Force  programmers  leave,  placing  a 
greater  burden  on  the  remaining  Air  Force  personnel  (9:51) . 

AFR  26-1,  Manpower:  Manpower  Policies  and  Pro¬ 
cedures  Comparative  Cost  Analysis,  stipulates  "Positions 
that  have  direct  combat  support  tasks  under  contingency  or 
War  Plans,  but  indirect  combat  support  tasks  in  peacetime 
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must  be  identified  as  Military  Essential." 
tions  require  Air  Force  programmers,  then  the  only  remain¬ 
ing  resource  available  to  the  organization  for  additional 
programming  or  maintenance  will  be  the  contract  programmer 
(10:64) . 

Section  IV 

Documentation .  Documentation  of  software  is 
another  tool  that  has  had  tremendous  influence  on  software 
maintenance.  Software  documentation  is  the  single  most 
effective  tool  for  software  maintenance  (14:1;  32:13; 
8:132,134;  1:42).  Studies  have  shown  that  documentation 
is  not  being  accomplished  for  all  programs  (4:115).  It 
takes  time  in  the  development  stage  to  write  good  documents 
tion.  That  time  and  the  resources  involved  could  be  used 
to  develop  additional  software.  The  trend  in  both  industry 
and  government  has  been  for  management  and  the  programming 
office  to  set  documentation  aside  (29:5). 

Since  maintenance  programmers  depend  heavily  on 
the  documentation  to  become  familiar  with  a  system  and 
since  maintenance  costs  are  much  greater  whan  development- 
costs,  software  documentation  can  be  considered  an  invest¬ 
ment  that  saves  time  and  resources  (32:13;  13:74).  Auto¬ 
mated  software  documentation  programs  are  available  but  do 
not  provide  sufficient  information  for  maintenance  per¬ 
sonnel.  However,  some  documentation  is  better  than  none 
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as  long  as  it  is  current  (8:142) .  Documentation  must  be 
updated  when  software  changes  are  made  or  the  documenta¬ 
tion  becomes  obsolete.  Worse  yet,  unrevised  documentation 
can  be  misleading  and  thus  further  increase  the  rate  of 
decay  and  time  needed  for  software  modification  (33:23). 

Software  Quality  Assurance.  Software  quality 
assurance  has  received  a  great  deal  of  publicity  in  the 
1980s  within  industry.  Software  quality  assurance  is 
important  in  the  area  of  maintenance  because  it  emphasizes 
developing  products  that  perform  correctly  when  turned  over 
to  the  user.  This  means  the  "corrective"  errors  will  be 
reduced  for  the  maintenance  programmers.  Software  quality 
assurance  begins  with  the  premise  that  certain  qualities 
are  desired  in  a  finished  product.  Within  the  Air  Force, 
reliability  and  maintainability  are  highest  on  the  software 
quality  assurance  priority  list.  Software  users  need  to 
understand  that  specific  qualities  or  "factors"  can  be 
built  into  software  but  only  at  certain  costs.  Some  soft¬ 
ware  qualities  build  on  other  qualities  while  others  con¬ 
flict  and  degrade  (21:24).  A  table  defining  these  quali¬ 
ties  and  how  they  affect  each  other  can  be  found  in  Appen¬ 
dix  G . 

No  one  wants  tc  develop  an  inferior  product  but 
quality  is  a  concept  that  involves  not  just  the  programmer 
but  everyone  from  the  program  manager  to  the  operator. 
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Software  quality  assurance  advocates  that  plans,  metrics/ 
standards,  and  controls/reviews  be  implemented  to  insure 
that  desired  quality  factors  are  assured  (27:1,3) . 

Conf igura tion  Management.  Configuration  manage¬ 
ment  is  the  management  of  change  (13:149).  It  consists  of 
four  functions:  identification,  configuration  control, 
status  accounting,  and  auditing  (24:34).  It  is  based  on 
a  concept  that  software,  tightly  controlled,  reduces  the 
choices  of  inadvertent  mistakes.  Mistakes  such  as  modifi¬ 
cation  of  software  without  an  associated  documentation 
update  or  inadequate  revision  testing  can  increase  future 
problems  for  the  software  and,  in  turn,  the  time  and 

resources  to  fix  it  (13:149). 

In  terms  of  testing,  configuration  management  deter 

mines  how  much  time  and  money  will  be  spent  "debugging"  the 
errors.  This  is  due  to  the  error  discovery  cost  rate.  As 
efforts  are  made  to  find  errors  many  will  be  found  early , 
but  it  takes  longer  and  longer  to  find  the  remaining 
errors.  See  Figure  2-1  (13:61)  . 


The  first  95  percent  of  the  errors  may  cost  little  in 
terms  of  dollars  and  time.  But  the  last  5  percent  of  those 
errors  may  cost  more  than  the  first  95  percent  (15:41). 

See  Figure  2-2  (5:40). 


Fig.  2-2.  Increase  in  Cost-to-Fix  or  Change 

Software  Throughout  Life  Cycle  (5:40) 

A  decision  has  to  be  made  as  to  when  testing  stops 
and  production  begins  (13:61)  .  Programs  will  reach  a  point 
where  they  have  become  obsolete  and  further  revision  is 
useless.  At  this  p  int  the  software  program  must  be 
rebuilt  (7:1).  Management  and  the  configuration  management 
shop  must  determine  how  to  best  manage  the  organization 
with  its  associated  responsibilities  and  resources  (31:36). 
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Use  of  software  cost  estimation  can  indicate  the  limit  of 
cost  effectiveness  for  use  or  rebuilding  (11:101-102). 

Since  management  determines  how  to  utilize  organi¬ 
zational  resources,  management  should  develop  procedures 
for  the  organization  to  follow.  This  can  begin  with  a 
simple  plan  of  who  does  what  function  in  information  sup¬ 
port  or  can  develop  into  a  highly  structured  organization 
with  offices  specializing  in  input/output  control,  tape 
libraries,  software  maintenance  shops,  etc.  (24:34). 

Metrics.  "You  cannot  manage  what  you  cannot 
measure"  is  a  basic  tenet  behind  the  development  of  soft¬ 
ware  metrics  (18:1).  Metrics  are  standards  or  measures 
that  can  be  applied  to  measure  the  effectiveness  and  effi¬ 
ciency  of  software  and  related  factors  such  as  testing  and 
documentation.  The  National  Bureau  of  Standards  has  listed 
several  areas  of  software  maintenance  metrics.  Appendix  I 
contains  these  lists. 

Relating  this  to  software  cost  estimation, 

Henderson  and  Sullivan  stated  that  "you  can't  measure  what 
you  don't  keep  data  on."  Not  only  should  the  metrics  be 
developed  and  used  but  the  information  derived  should  be 
archived  for  future  reference. 

Off-the-shelf  Software.  With  the  growing  costs 
in  software  development  and  maintenance,  many  companies 
are  turning  to  commercially  available  or  "off-the-shelf" 
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software  to  meet  specific  organizational  needs  (17:744). 
This  type  of  software  can  be  purchased  from  the  computer 
hardware  manufacturer  or  from  a  third  party  vendor.  "Off- 
the-shelf"  software  has  benefits  and  handicaps.  On  the 
beneficial  side,  "off-the-shelf"  software  usually  costs 
less  than  developing  and  maintaining  unique  software. 
"Off-the-shelf"  software  is  available  now  and  not  after 
several  months  of  development  work.  On  the  negative  side, 
"off-the-shelf"  software  may  not  meet  all  the  needs,  may 
not  be  available  for  the  organization's  computer,  may  not 
have  training  available  in  its  use,  and  may  not  be  main¬ 
tainable  due  to  vendor  policy  such  as  restricted  documenta¬ 
tion  (17:744). 
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III.  Research  Methodology 

This  chapter  will  discuss  the  procedures  used  to 
collect  and  analyze  available  information  in  order  to 
satisfy  the  research  objectives  proposed  in  Chapter  I. 

» 

Specifically,  it  focuses  on  data  collection  by  means  of  a 
literature  review  and  a  statistical  analysis  of  the  infor¬ 
mation  contained  in  the  Air  Force  Information  Systems 
Designator  (ISD)  database. 

AFR  700-19  Database 

To  adequately  assess  the  software  costs  in  the 

< 

Air  Force  several  methods  were  proposed.  The  first  method 
involved  contacting  every  command  for  information  on  costs. 
This  was  deemed  inappropriate  due  to  the  time  constraints 
of  the  thesis.  The  second  method  examined  the  Major  Com¬ 
mand  Information  System  Plan  (MISP)  required  by  AFR  700-2 
for  Air  Force  commands.  Research  at  the  historical 
archives  at  HQ  Air  Force  Logistics  Command  (AFLC)  of 
several  command's  MISPs  shewed  that  maintenance  costs  are 
not  recorded  at  this  time.  Maintenance  costs  are  not 
required  in  current  MISPs.  The  third  method  involved  find¬ 
ing  an  established  database  with  this  information.  The 
Data  Item  Designator  database  at  Air  Force  Systems  Command, 
Electronic  Systems  Division  (AF5C/ESD)  was  examined.  This 
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database  covers  "embedded"  software  rather  than  the  ADPE 
software  systems  used  in  large-scale  data  processing 
computers  being  examined  in  this  study.  One  AF  database 
was  finally  found.  AFR  700-19  established  the  Information 
Systems  Designator  (ISD)  database  maintained  by  the 
Standard  Information  Systems  Center  (SISC) ,  a  specialized 
computer  center  for  the  Air  Force  Communications  Command. 

The  AF  ISD  database  was  created  in  the  1960s  to  provide 
specific  information  on  AF  software  for  large  automatic 
data  processing  (ADP)  systems.  At  that  time,  the  database 
was  a  part  of  the  AF  300  series  regulations  and  known  as 
the  Data  Systems  Designator  (DSD)  Database  and  maintained 
by  Data  System  Design  Office  headquartered  at  Gunter  AFS, 
Alabama.  The  purpose  for  the  database  was  to  share  part, 
if  not  all,  of  the  software  between  organizations  with 
similar  software  requirements.  This  shared  software  would 
reduce  duplication  of  effort  and  software  costs.  All  AF 
commands  are  required  to  report  this  information  for  their 
active  systems.  Today,  this  database  includes  software  of 
all  types  of  application  systems  except  embedded  and  classi¬ 
fied.  SISC  at  Gunter  AFS,  Alabama  is  now  responsible  for 
publishing  and  managing  this  information  under  Air  Force 
Regulation  700-19. 

The  procedure  works  in  the  following  manner.  After 
defining  a  needed  software  capability,  an  organization 
would  examine  their  copy  of  the  ISD  database  for  any 
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possible  compatible  software  systems.  The  organization 
would  then  go  directly  to  the  owner  of  the  functionally 
equivalent  software  listed  in  the  database  to  obtain  a 
copy.  If  both  organizations  use  the  software,  then  the 
second  organization  must  also  register  with  SISC.  If 
nothing  is  found,  the  organization  can  develop  the  soft¬ 
ware  for  their  use  and  then  register  it  with  SISC  for  the 
benefit  of  other  organizations. 

Specific  Data  Fields  Used 

SISC  assigns  a  unique  identifier  called  an  ISD 
for  each  software  system  submitted  to  this  database.  The 
separate  ISDs  list  primarily  a  description  of  what  the 
software  does.  In  addition,  many  fields  are  included  in 
the  record  stating  language  used,  hardware  used,  user 
command,  development  costs,  maintenance  costs  and  more. 

A  listing  of  all  the  fields  contained  in  this  database  is 
found  in  Appendix  D.  After  studying  this  database  for 
analysis,  specific  fields  were  examined  for  this  study. 
These  fields  are: 

1.  YEAR — this  data  element  lists  the  year  the 
software  was  placed  in  operation. 

2.  CONTRACTOR — this  field  is  either  a  one,  two 

or  a  blank.  One  means  the  software  was  at  least  50  percent 
contractor  developed.  Two  means  the  software  is  commer¬ 
cially  developed  or  "off-the-shelf"  software.  A  blank 
means  the  software  was  developed  within  the  Air  Force. 


3.  COMMAND — this  field  states  the  primary  user 

command. 

4.  LANGl — this  field  states  the  primary  program¬ 
ming  language  used.  In  several  large  software  systems, 
more  than  one  language  was  used.  Since  the  records  did 
not  break  down  the  size  or  cost  by  language,  only  the 
first  language  would  be  used  for  this  category. 

5.  LINES — this  field  lists  the  number  of  lines 
of  code  in  the  program (s)  . 

6.  DEVCOST — this  data  field  gives  the  development 
cost  of  the  system  when  it  was  put  into  operation. 

7.  MANCOST— this  final  data  field  lists  the 
accumulated  costs  for  maintenance  work  since  the  software 
was  placed  into  operation.  This  field  is  updated  yearly 
although  no  field  indicated  the  last  year  of  update. 

Computer  Support 

The  ISD  database  is  designed  to  be  a  report 
generator.  The  records  and  fields  are  variable  length. 
SISC  does  not  perform  any  statistical  analysis  on  this 
data;  it  is  used  only  to  give  a  description  of  that  ISD 
record.  Since  this  study  examined  the  size  and  costs  of 
the  software,  verification  of  the  costs  was  desired.  The 
data  is  required  to  be  sent  to  SISC  from  the  commands  and 
changes  are  to  be  updated  yearly.  Although  SISD  verified 
that  the  data  sent  for  this  study  was  the  same  as  their 
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database,  all  attempts  to  verify  this  data  from  a  second 
source  were  unsuccessful.  It  cannot  be  assumed  that  the 
data  is  accurate  neither  can  it  be  assured  to  be  in  error. 
Therefore,  the  results  of  this  study  art  only  as  good  as 
the  data  available. 

In  order  to  accomplish  statistical  analysis, 
another  database  was  built  on  the  AFIT  Classroom  Support 
Computer  (CSC)  using  the  information  extracted  from  the 
ISD  database.  The  CSC  is  a  Digital  Equipment  Corporation 
VAX  computer  with  the  Virtual  Memory  Storage  (VMS) 
operating  system.  This  computer  system  was  used  because 
of  the  availability  of  the  "SAS"  statistical  package. 

"SAS"  is  a  copyrighted  statistical  package  that  allows  data 
to  be  analyzed  and  reported  in  several  different  ways  in 
the  same  computer  run. 

One  of  the  first  steps  in  the  analysis  required  a 
frequency  count  of  the  number  of  occurrences  of  the  pro¬ 
gramming  languages,  the  various  commands,  and  the  number  of 
programs  developed  under  contract  or  off-the-shelf  soft¬ 
ware.  It  was  decided  that  to  be  a  representative  sample, 
thirty  examples  of  the  field  being  examined  must  be 
present.  A  copy  of  the  statistical  program  is  found  in 
Appendix  B . 


Research  Objective  One 

The  first  objective,  what  has  been  tdie  change  in 
AF  development  costs  over  time,  will  be  answered  by 
analyzing  the  difference  in  the  cost  per  line  of  code  over 
time.  After  being  examined  as  an  AF  total  per  year,  the 
data  will  be  analyzed  according  to  the  programming  language 
used,  by  the  AF  user  command  and,  finally,  contractor  soft¬ 
ware  development  costs.  Examining  the  total  Air  Force 
development  costs,  1342  examples  were  available  from  the 
years  1955  through  1986.  The  number  of  lines  of  code 
developed  during  a  specific  year  was  divided  into  the  total 
development  costs  for  that  year. 

Total  Development  Cost/Total  Number  of  Lines 
=  Cost  per  Line 

Most  of  the  data  was  studied  by  year.  If  a  record 
did  not  list  a  specific  year  in  the  year  field,  that 
record  was  deleted  from  further  analysis  in  the  development 
cost,  since  it  could  not  be  placed  into  a  specific  year 
category.  This  resulted  in  254  records  being  deleted  from 
further  analysis. 

Air  Force  commands  were  required  to  report  the 
development  and  maintenance  costs  with  their  ISD  informa¬ 
tion.  If  actual  costs  were  not  available,  AFR  700-19  pro¬ 
vided  a  formula  for  estimating  the  costs  of  $20  per  line 
of  code  for  development  costs  and  $80  per  line  of  code  for 


maintenance  costs.  There  is  no  indication  in  the  records 
of  whether  the  actual  costs  or  the  AFR  700-19  formulated 
costs  were  reported  to  SISC.  Although  some  of  the  ISD 
records  may  list  costs  above  the  actual  costs,  just  as 
likely  some  costs  would  be  below  the  actual  costs. 

Averaged  across  the  1600+  records,  it  is  likely  that  the 
overall  costs  per  line  would  be  close  to  the  true  actual 
cost  of  AF  software  per  line. 

The  average  cost  per  line  of  code  for  each  command 
is  generated  by  the  total  cost  for  that  command  divided 
by  the  total  number  of  lines  of  those  systems.  This  same 
procedure  will  be  performed  for  each  programming  language 
used  to  give  the  average  AF  cost  by  year  per  line  of  code 
for  that  language. 

Research  Objective  Two 

The  second  objective,  have  these  development  costs 
shown  the  same  increase  independent  of  the  AF  command  or 
the  programming  language  used,  will  build  on  the  first 
objective.  Where  objective  one  looked  at  actual  numbers 
and  cost  per  line,  objective  two  will  look  at  total  dollar 
trends.  SAS  will  be  used  to  build  charts  showing  develop¬ 
ment  costs  compared  over  time.  The  year-by-year  progres¬ 
sion  will  be  shown  on  the  same  chart.  Charts  showing  com¬ 
bined  AF,  unique  programming  language,  and  specific 
commands  will  be  built.  The  detailed  numbers  will  be 
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available  for  in-depth  comparison  of  the  cost  ratios  com¬ 
pared  to  lines  of  code. 

Research  Objective  Three 

Objective  three,  what  have  been  the  costs  of  soft¬ 
ware  maintenance  for  these  systems,  is  similar  to  objective 
one  except  that  maintenance  costs  will  be  examined  instead 
of  development  costs.  Again,  the  categories  will  be  the 
cost  per  line  of  maintenance  work  as  a  whole  within  the 
AF,  then  the  programming  language  used,  the  user  commands 
and,  finally,  contracted  software.  The  analysis  will  then 
proceed  year  by  year  to  look  for  any  indication  of  rising 
or  lowering  at  an  unusual  rate. 

Total  Accumulated  Maintenance  Costs/Total  Number  of  Lines 

*  Cost  per  Line 

If  any  year  showed  unusual  figures,  a  table  list¬ 
ing  the  specific  category  and  year  will  provide  further 
information  to  clarify  the  peculiarities. 

From  this  analysis,  the  cost  of  Air  Force  software 
maintenance  should  be  found  and  a  comparison  against  con¬ 
tractor  costs  can  show  which  is  the  most  economical.  Since 
the  industry  average  costs  are  already  known  from  the 
literature  search,  analysis  of  the  Air  Force  costs  can  also 
provide  a  comparison  in  several  ways,  between  the  Air  Force 
versus  industry  and  contracting  versus  industry. 
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As  with  the  first  and  second  objectives,  the  fre¬ 
quency  count  will  be  used  to  decide  on  the  specific  lan¬ 
guages  and  commands  to  examine.  For  additional  comparison 
in  the  event  of  unusual  findings,  all  remaining  languages 
will  be  placed  in  an  "other"  language  category  and  the 
remaining  commands  will  be  placed  in  an  "other"  command 
category  for  analysis  and  comparison  against  the  specific 
languages  and  commands. 

Research  Objective  Four 

Objective  four,  have  maintenance  costs  shown  the 
same  increase  independent  of  the  AF  command  or  programming 
language  used,  will  be  examined  in  a  similar  fashion  to 
objective  two.  Here,  the  purpose  is  not  to  look  at  the 
efficiency  or  inefficiency  in  the  maintenance  costs,  but  to 
look  at  the  total  volume  effort — the  quantity.  The  total 
maintenance  effort  will  be  examined  within  the  Air  Force 
year  by  year.  This  will  be  sorted  by  languages  for  analy¬ 
sis  then  resorted  and  analyzed  by  commands  to  see  if  any 
particular  unusual  costs  are  found.  Finally,  the  con¬ 
tractor  software  will  be  analyzed  as  a  whole  and  by  year. 
This  information,  placed  on  a  graph,  should  show  total 
costs  and  trends  in  maintenance  cost. 

Research  Objective  Five 

The  final  objective,  what  differences  are  shown 
between  development  and  maintenance  costs,  analyzed  as 
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were  the  previous  objectives,  will  show  whether  Air  Force 
maintenance  costs  are  greater  than  the  development  costs 
for  the  same  systems.  If  this  proves  to  be  the  case,  then 
this  would  show  a  trend  similar  to  that  in  industry. 

The  literature  search  found  industry  software  maintenance 
costing  at  least  as  much  as  the  development  costs.  The 
Air  Force  costs  for  development  and  maintenance  may  not 
equal  industry's,  but  if  the  percentage  differences  between 
Air  Force  development  to  maintenance  is  similar  to  indus¬ 
try'  s  development  to  maintenance  ratio  then  the  same 
problems  occurring  in  industry  may  be  occurring  in  the 
Air  Force. 

If  the  analysis  of  the  data  shows  that  the  Air 
Force  software  maintenance  is  similar  to  industry's  then 
the  methods,  techniques,  and  problems  in  industry  may  be 
the  same  as  the  Air  Force's.  Improvements  in  industry  may 
be  used  to  reduce  the  maintenance  costs  in  the  Air  Force 
well. 


IV.  Research  Observations 


To  better  understand  the  costs  of  Air  Force  soft¬ 
ware,  analysis  of  the  cost  and  size  data  from  a  computer 
software  database  was  categorized  and  totaled  using  the  SAS 
statistical  package.  The  details  and  analysis  are  pre¬ 
sented  here. 

To  begin  the  analysis  of  the  database,  the  SAS 
frequency  option  was  used  to  categorize  the  data.  The 
frequency  option  gave  a  count  for  each  occurrence  in  each 
field.  Table  4-1  shows  the  different  commands  represented 
with  their  respective  frequency  count.  Thirteen  commands 
contain  enough  samples  to  be  selected  for  individual  analy¬ 
sis.  The  remaining  commands  were  grouped  into  an  "other" 
command  category  for  comparison.  The  thirteen  commands 
are  Air  Force  Accounting  and  Finance  Center  (AFAFC) ,  Air 
Force  Communications  Command  (AFCC) ,  Air  Force  Logistics 
Command  (AFLC) ,  Air  Force  Systems  Command  (AFSC) ,  Air 
Training  Command  (ATC) ,  Air  University  (AU) ,  Data  Systems 
Design  Organization  (DSDO) ,  Electronics  Security  Command 
(ESC) ,  Military  Airlift  Command  (MAC) ,  Pacific  Air  Forces 
(PACAF) ,  Strategic  Air  Command  (SAC) ,  Tactical  Air  Command 
(TAC) ,  and  United  States  Air  Forces  Europe  (USAFE) . 
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TABLE  4-1 


COMMAND  FREQUENCY  COUNT 


! 

I 

I 

I 


I 

I 

I 


Command 

Frequency 

Command 

Frequency 

AAC 

11 

ANGSC 

1 

AFAA 

1 

ATC 

68 

AFAFC 

27 

AU 

48 

AFCAC 

1 

DCA 

13 

AFCC 

240 

DLA 

1 

AFCOMS 

2 

DNA 

1 

AFCOS 

1 

DSDO 

30 

AFDSDO 

1 

ESC 

49 

AFESC 

1 

JCS 

2 

AFIS 

16 

JDA 

1 

AFISC 

5 

JDSSC 

4 

AFLC 

394 

MAC 

153 

AFLMC 

6 

MMSSA 

1 

AFMEA 

1 

PACAF 

62 

AFMPC 

6 

SAC 

130 

AFMSC 

11 

SPACECMD 

20 

AFMSMET 

1 

TAC 

92 

AFOTEC 

2 

TACC 

2 

AFRES 

4 

TPSC 

21 

AFSC 

96 

USAF 

5 

AFWL 

1 

USAFA 

5 

ANG 

1 

USAFE 

63 

39 
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Table  4-2  shows  the  different  languages  and  their  frequency 
count.  The  languages  were  handled  in  a  similar  fashion  to 
the  commands.  Three  languages.  Assembler,  FORTRAN,  and 
COBOL,  had  enough  samples  for  each  individual  study.  The 
remaining  programming  languages  were  grouped  into  an 
"other"  language  category. 

TABLE  4-2 

LANGUAGE  FREQUENCY  COUNT 


Research  Objective  One 

What  has  been  the  increase  in  development  costs 
over  time? 

To  compare  these  costs,  two  additional  standards 
were  developed  in  addition  to  the  industry  average  men¬ 
tioned  in  Chapter  II.  The  total  AF  cost  per  line  was 
calculated  from  the  total  number  of  lines  and  total  devel¬ 
opment  costs.  The  second  standard  was  developed  for  each 


language  or  command.  In  both  cases  the  "years  developed" 
was  ignored.  This  table  is  shown  in  Table  4-3.  Develop¬ 
ing  these  standards  using  the  total  records  regardless  of 
year  is  important  to  the  database  because  several  hundred 
records  lacked  "year  developed"  preventing  these  records 
from  being  categorized  for  analysis.  With  the  total 
records  for  development  costs  calculated,  the  records  could 
be  categorized  and  compared  against  these  three  standards 
(industry,  AF,  category)  over  time.  Tables  were  built  for 
tne  languages  over  time  and  commands  over  time.  These  can 
be  viewed  in  Appendix  E.  Graphs  showing  the  cost  per  line 
changes  over  the  various  years  can  be  found  in  Appendix  C. 

AF  Total  Cost.  Examining  the  cost  per  line  of 
code,  the  AF  developed  cost  per  line  of  code  is  found  to 
fall  below  the  industry  cost  range  of  $50  to  $200.  This 
cost  was  calculated  by  dividing  the  total  development  cost 
by  the  total  number  of  lines. 

$1,241,355,956  /  653,760,528 
=  $1.90  per  Line  of  Code 

The  calculations  on  the  cost  per  line  year  by  year  showed 
the  cost  in  the  high  teens  until  1972,  at  which  time  it 
dropped  into  single  digits.  One  exception  occurred  in 
1962  with  $233.60  per  line  of  code.  Examining  this 
exception,  it  was  found  that  the  total  development  cost  was 
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not  high  compared  to  the  other  years  but  the  number  of 
lines  of  code  was  unusually  low  which  boosted  che  price 
per  line.  Although  SISC  verified  their  data,  this  cost 
was  caused  by  a  large  HQ  MAC  development  cost.  The  other 
two  commands  that  showed  development  costs  in  1962,  AFCC 
and  AFLC,  averaged  §20  and  $18.68  respectively.  This  means 
that  excluding  the  HQ  MAC  software  system,  1962  followed 
the  average  with  the  other  years,  1955  to  1971. 

Programming  Languages .  Examining  the  languages 
next,  Assembler,  COBOL,  and  FORTRAN  had  enough  examples  to 
list  each  as  a  unique  category.  All  other  languages  were 
grouped  together  into  an  "Other"  category.  Refer  to 
Appendices  C  and  E  for  the  related  graphs  and  tables. 

The  Assembler  category  had  fifty-eight  records 
through  1985  and  only  eight  of  the  fifty-eight  assembler 
programs  listed  as  operational  before  1975.  The  total 
Assembler  cost  per  line  of  code  was  $2.70.  Generally,  the 
cost  per  line  was  higher  than  the  Air  Force  average  of 
$1.90  per  line  of  code. 

The  COBOL  language  was  the  largest  category, 
with  1185  records,  of  the  four  language  categories 
observed.  Dividing  the  total  development  cost  by  the  total 
number  of  lines  gave  an  average  development  cost  per  line 
of  $11.08.  The  development  cost  per  line  for  years 
1955  to  1961  rose  above  $20  per  line.  Looking  year  by 
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year  for  thirty-one  years,  the  costs  never  rose  above 
$34.98  per  line.  Again,  this  is  below  the  $50  to  $200 
per  line  range  in  industry. 

The  FORTRAN  programming  language  had  197  records 
with  a  total  cost  per  line  average  of  $.47.  Thirty-one 
records  were  removed  from  further  analysis  because  of 
insufficient  data  in  the  year  field.  The  166  remaining 
records  covered  the  period  from  1962  until  1984.  Comparing 
against  the  total  AF  yearly  average,  programs  written  in 
FORTRAN  were  higher  overall  than  the  yearly  average.  A 
major  factor  for  the  low  overall  development  cost  per  line 
is  due  to  one  system  written  in  FORTRAN  in  1972  by  AFCC. 
This  command  had  a  system  containing  500,000,000  lines  of 
code  at  a  cost  of  $100,000,000.  If  this  system  is  removed 
from  the  calculations,  the  FORTRAN  cost  per  line  rises  to 
$15.95. 

The  final  category  examined  is  the  "Other"  cate¬ 
gory.  Here  all  the  languages  that  did  not  contain  at  least 
thirty  representative  systems  were  grouped  together  for 
analysis . 

The  "Other"  category  contained  128  systems  with  a 
total  cost  per  line  of  $5.23.  This  is  almost  three  times 
higher  than  the  AF  total  average  of  $1.90  per  line.  Com¬ 
paring  year  by  year,  the  "Other"  category  had  higher 


overall  costs. 


Command.  The  development  costs  were  also  examined 


by  command.  Thirteen  commands  contained  enough  systems  to 
be  examined  separately  and  the  remaining  thirty-one  repre¬ 
sentative  commands  were  placed  in  the  "Other"  category 
for  a  combined  analysis.  Refer  to  Appendices  C  and  E  for 
the  graphs  and  tables. 

The  first  command  examined,  AFAFC  with  twenty- 
seven  examples,  had  a  total  average  cost  per  line  of  $16.71 
which  is  above  the  total  Air  Force  average. 

AFCC  had  240  systems  registered  in  the  database. 

The  average  development  cost  per  line  was  $.31.  This  was 
the  lowest  command  total  development  cost  per  line  and  far 
below  both  the  AF  and  industry  cost  per  line.  A  major 
reason  for  the  low  command  total  cost  per  line  is  the 
500,000,000  line  system  developed  in  1972  at  a  cost  of 
$100,000,000  and  a  50,000,000  line  system  developed  for 
$120,000  in  1975.  These  two  systems  had  a  major  effect 
not  only  on  AFCC  but  on  the  AF  yearly  and  total  cost  per 
line.  Twenty-five  years  were  represented  and  during  most 
of  those  years,  AFCC  was  below  the  AF  yearly  cost  per  line. 

AFLC's  file  contained  394  systems  which  was  the 
most  for  any  command.  Only  thirteen  of  AFLC's  systems  were 
deleted  because  of  an  incomplete  year  field.  AFLC  had  a 
total  average  of  $11.61  per  line  which  is  well  above  the 
AP  total  average.  Examining  each  year  found  that  AFLC 
exceeded  the  AF  yearly  average  fifteen  times,  was  below 
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the  AF  yearly  average  most  years,  but  was  always  below 
industry  costs. 

AFSC  had  over  2.5  million  lines  of  code  in  ninety- 
six  systems  giving  an  average  cost  of  $19.52  per  line  for 
development.  During  a  twenty-year  period,  ninety-two 
systems  were  listed  as  active. 

ATC  excluded  only  three  of  its  sixty-eight  systems 
due  to  a  mission  year  field.  The  total  average  cost  per 
line  was  $1.77  which  is  slightly  below  the  AF  average  cost 
per  line.  ATC  costs  year  by  year  were  overall  below  the 
AF  average. 

AU  had  forty-eight  systems  with  an  average  cost 
of  $2.77  per  .\ine  which  is  slightly  higher  than  the  AF 
average.  Only  thirty  of  the  systems  indicated  a  specific 
year  with  a  time  span  of  sixteen  years.  AU's  year-by-year 
costs  were  below  the  AF  yearly  averages. 

DSDO,  now  known  as  SISC,  had  thirty  systems  listed 
covering  a  five-year  period.  The  total  average  per  line 
of  code  cost  is  $19.41  or  ten  times  greater  than  the  AF 
total  per  line  cost.  Only  five  of  the  thirty  systems 
listed  a  year  in  the  required  field.  SISC  year-by-year 
averages  also  exceeded  the  AF  year-by-year  averages. 

The  next  command  examined  was  ESC  with  forty-nine 
computer  software  systems  with  a  low  total  per  line  average 
cost  of  $.68.  Only  thirty-eight  of  the  systems  could  be 
analyzed  by  year  due  to  missing  data  in  the  year  field. 
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ESC' 8  systems  are  relatively  new  suggesting  that  the  cost 
would  be  higher  than  earlier  systems  developed  elsewhere. 
This  was  not  the  case.  Every  year  lists  a  lower  yearly 
average  per  line  cost,  except  1982  with  $5.37,  than  the 
AF  total  average  per  line  cost. 

MAC  had  the  third  largest  sample  with  153  systems 
averaging  $16.65  per  line  of  code.  This  is  almost  nine 
times  greater  than  the  AF  average  cost.  Only  five  MAC 
systems  lacked  a  year  identifier  with  the  remaining  systems 
covering  a  twenty-year  period.  Of  these  twenty  years 
analyzed,  most  were  above  the  AF  total.  The  yearly  average 
cost  per  line  was  skewed  in  1967  due  to  a  large  development 
cost  on  one  system.  If  that  system  is  removed  from  the 
calculations,  the  per  line  cost  for  that  year  is  reduced 
to  $20.  The  same  situation  occurs  again  in  1970  with 
similar  results  when  changed.  Overall,  the  per  line  costs 
are  well  within  the  range  of  AF  costs,  although  they  are 
below  the  average  in  all  cases  for  industry. 

PACAF  was  unusual  to  analyze  because  it  had  sixty- 
two  systems  registered  but  only  twenty-two  of  them  listed 
the  year  developed.  Of  these  twenty-two,  fifteen  were 
lacking  development  cost  and  two  lacked  the  number  of  lines 
in  the  system.  With  the  remaining  information  for  the 
command,  the  total  average  development  cost  per  line  was 
calculated  to  be  $1.67.  This  is  slightly  below  the  AF 
total  average  cost  per  line.  Every  one  of  the  four  years 
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available  for  comparison  were  below  the  AF  yearly  average 
cost  per  line.  All  four  were  below  three  dollars  per  line. 

SAC  was  examined  next.  The  130  systems  covered 
1963  through  1985,  although  twenty- two  systems  lacked 
appropriate  information  to  categorize  by  year.  SAC's  total 
average  cost  per  line  is  $16.65.  This  is  nearly  nine  times 
greater  than  the  equivalent  AF  cost  per  line.  For  the 
twenty  years  examined,  SAC's  cost  per  line  was  above  the 
AF  average  yearly  cost  most  times.  Only  one  year,  1972, 
fell  in  the  same  range  as  industry's. 

TAC  had  ninety-four  systems  listed  which  averaged 
$17.51  per  line.  This  is  over  nine  times  greater  than  the 
AF  average.  Only  seventy-one  of  these  systems  listed  the 
year  developed  covering  the  thirteen-year  period.  These 
thirteen  years  were  mostly  above  the  average. 

USAFE  is  the  final  unique  command  examined.  It 
contained  sixty-three  records  which  gave  a  total  average 
cost  per  line  of  $5.03.  Only  fifty-six  records  had  the 
year  indicated.  These  records  covered  twelve  years.  The 
USAFE  yearly  total  cost  per  line  exceeded  the  AF  yearly 
total  cost  per  line  about  half  the  time. 

Finally,  the  remaining  commands  were  grouped 
together  under  a  heading  of  "Other."  This  added  up  to 
126  records  covering  twenty-nine  commands.  Their  combined 
records  gave  a  total  average  per  line  cost  cf  $8.68,  over 
four  and  one-half  times  larger  than  the  AF  total  average 
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cost  per  line.  The  records  covered  a  total  of  twenty-one 
years  from  1965  until  1985.  During  this  period,  the 
"Other"  category  exceeded  the  AF  yearly  average  cost  per 
line  about  half  the  time. 

Contractor  Developed  Software.  Examining  the  costs 
of  development  for  contractor-developed  software,  sixty- 
seven  records  in  the  database  were  at  least  50  percent 
contractor  developed.  They  give  a  total  average  develop¬ 
ment  cost  per  line  of  $29.25  which  is  almost  seventeen 
times:  the  AF  equivalent.  However,  this  is  below  industry 
average  costs  of  $50  to  $200  per  line  of  code.  It  would 
be  expected  that  contractors  would  account  for  their  costs 
more  along  the  lines  of  industry  than  the  AF. 

Research  Objective  Two 

Have  these  development  costs  shown  the  same 
increase  independent  of  the  AF  command  of  the  programming 
language  used? 

To  examine  the  rise  in  software  development  costs, 
the  costs  were  graphed  by  year  and  placed  against  a 
standard  y-axis  of  $200,000,000.  Looking  at  the  combined 
AF  graph  shows  a  cyclical  but  steadily  rising  software 
development  cost. 

Programming  Language.  Looking  at  the  various 
languages,  Assember  did  not  really  show  enough  of  a 
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pattern  to  draw  any  conclusions.  See  Appendix  F.  The 
chart  indicates  that  assembler  costs  are  declining.  This 
may  follow  with  the  emphasis  towards  use  of  higher  order 
languages  in  computer  systems. 

COBOL  systems  were  well- represented  and  showed  the 
same  pattern  as  the  AF  chart,  cyclical  but  steadily  rising. 
FORTRAN  indicated  just  the  opposite.  Costs  are  cyclical 
but  diminishing  which  may  indicate  either  reduced  cost  per 
line  of  FORTRAN,  smaller  programs,  or  less  use  of  FORTRAN 
for  system  development.  The  "Other"  category  shows  only  a 
small  period  which  may  not  be  a  full  cycle.  These  costs 
appear  to  be  diminishing  as  well.  See  Appendix  F. 

Command.  The  ccmmands  show  more  of  a  continuous 
flow  in  development  costs  rather  than  a  cyclical  rise. 

AFCC,  SAC,  and  TAC  do  show  occasional  peaks  but  most,  like 
AFLC ,  indicate  a  budget  with  computer  systems  being  devel¬ 
oped  when  they  can  be  fitted  into  that  budget.  Air  Force 
organizations  have  requested  increasing  software  support 
over  the  past  thirty  years.  Budgets  were  increased  accord¬ 
ingly  which  allowed  ADP  offices  to  provide  the  resources 
needed.  Large  projects  though  were  multi-year  development 
projects  and  the  way  the  costs  were  recorded  only  showed  up 
in  the  AFR  700-19  database  when  the  computer  systems  were 
placed  into  operation.  This  in  turn  showed  up  as  peaks  on 
the  graphs. 
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Research  Objective  Three 

What  have  been  the  costs  of  software  maintenance 
for  these  systems? 

As  with  research  objective  one,  objective  three 
began  comparisons  by  first  developing  additional  standards. 
The  total  AF  cost  per  line  was  calculated  from  the  total 
number  of  lines  and  total  maintenance  costs.  The  second 
standard  was  developed  for  each  language  or  command.  In 
both  cases,  the  "years  developed"  field  was  ignored.  See 
Table  4-4.  This  is  important  to  the  database  because 
several  hundred  records  lacked  "year  developed"  preventing 
these  records  from  being  categorized  for  analysis.  Now 
the  records  could  be  categorized  and  compared  against  these 
three  standards  (industry,  AF,  category)  over  time. 

Specific  tables  were  built  for  the  languages  over  time  and 
commands  over  time.  Refer  to  Appendices  C  and  E. 


AF  Total  Cost.  Examining  the  cost  per  line  of 
code,  the  AF  accumulated  maintenance  cost  is  found  to  be 
much  lower  than  industry  averages  of  $50  to  $2000.  The 
total  number  of  lines  of  code  written  was  653,760,528. 
The  total  maintenance  costs  for  these  lines  was 
$1,489,294,836  or  $2.27  per  line  of  code. 


Programming  Languages.  Examining  the  languages 
next.  Assembler,  COBOL,  and  FORTRAN  had  enough  examples 
to  list  each  as  a  unique  category.  All  other  languages 
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were  grouped  together  into  an  "Other"  category.  The  graphs 
and  tables  are  found  in  Appendices  C  and  E  respectively. 

The  Assembler  category  had  fifty-eight  records 
through  1985  with  only  eight  of  the  fifty-eight  assembler 
programs  listed  as  operational  before  1975.  The  total 
Assembler  cost  per  line  of  code  was  $2.08.  Generally,  the 
accumulated  maintenance  cost  per  line  was  lower  than  the 
Air  Force  average  of  $2.27  per  line  of  code. 

The  COBOL  language  was  the  largest  category  of  the 
four  language  categories  observed  with  1185  records. 
Dividing  the  total  accumulated  maintenance  costs  by  the 
total  number  of  lines  gave  an  average  maintenance  cost  per 
line  of  $22.24. 

The  FORTRAN  programming  language  had  197  records 
with  a  total  per  line  average  of  $.25.  Thirty-one  records 
were  removed  from  further  analysis  because  of  insufficient 
data  in  the  year  field.  The  166  remaining  records  covered 
the  period  from  1962  until  1984.  Comparing  against  the 
total  AF  yearly  average,  programs  written  in  FORTRAN  were 
higher  than  the  yearly  average  thirteen  out  of  twenty-two 
years  and  lower  than  the  average  nine  years. 

The  final  category  examined  is  the  "Other"  cate¬ 
gory.  Here  all  the  languages  that  did  not  contain  at  least 
twenty- seven  representative  systems  were  grouped  together 
for  analysis.  A  listing  of  the  various  languages  can  be 
found  in  the  frequency  table  in  Table  4-2  (page  40)  . 
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The  category  "utilities"  or  "utility"  actually  contained 
many  different  languages  so  these  were  placed  into  the 
"Other"  category  despite  having  fifty-nine  representative 
systems.  The  "Other"  category  contained  128  systems  with 
a  total  cost  per  line  of  $2.99.  This  is  only  $.80  higher 
than  the  AF  total  average  of  $2.19  per  line. 

I 

I 

k 

Command.  The  maintenance  costs  were  also  examined 

by  command.  Thirteen  commands  contained  enough  systems  to 

I  be  examined  separately  and  the  remaining  thirty-two  repre- 

> 

sentative  commands  were  placed  in  the  "Other"  category  for 
a  combined  analysis.  Refer  to  Appendices  C  and  E  for  the 
I  charts. 

I  The  first  command  examined,  AFAFC  with  twenty-seven 

I  examples,  had  a  total  average  cost  per  line  of  $29.84  which 

H 

'  is  well  above  the  total  AF  average  of  $2.19.  During  the 

twelve  years  examined,  AFAFC  exceeded  the  yearly  AF  average 
maintenance  cost  per  line  for  eight  of  those  years  and 
the  remaining  years  were  below  the  AF  yearly  average. 

AFCC  had  240  systems  registered  in  the  database. 

;  The  average  maintenance  cost  per  line  was  $.21.  This  was 

■  the  lowest  command  total  accumulated  maintenance  cost  per 

S  line  and  far  below  both  the  AF  and  industry  cost  per  line. 

|  AFLC ' s  file  contained  394  systems  which  was  the 

(  most  for  any  command.  Only  thirteen  of  AFLC's  systems  were 

|  deleted  because  of  an  incomplete  year  field.  AFLC  had  a 


total  average  of  $17.75  per  line  which  is  well  above  the 
AF  total  maintenance  average  of  $2.19. 

AFSC  was  the  next  command  examined.  This  command 
had  over  2.5  million  lines  of  code  in  ninety-six  systems 
giving  an  average  maintenance  cost  of  $21.82  per  line. 
During  a  twenty-year  period,  ninety-two  systems  were  listed 
as  active.  The  AF  yearly  average  cost  per  line  was 
exceeded  about  half  the  time. 

ATC  excluded  only  three  of  its  sixty-eight  systems 
due  to  faulty  year  fields.  The  total  average  cost  per 
line  was  $4.10  which  is  almost  double  the  AF  average  main¬ 
tenance  cost  per  line.  ATC  was  below  the  AF  average  most 
of  the  time. 

AU  had  forty-eight  systems  with  an  average  main¬ 
tenance  cost  of  $4.25  per  line  which  is  almost  double  the 
AF  average.  Only  thirty  of  the  systems  indicated  a  spe¬ 
cific  year  with  a  time  span  of  sixteen  years.  Most  years 
were  below  the  AF  yearly  total  average. 

SISC  had  thirty  systems  listed  covering  a  five- 
year  period.  The  total  average  per  line  of  code  cost  is 
$70.14  or  thirty- two  times  greater  than  the  AF  total  per 
line  cost.  Only  five  of  the  thirty  systems  listed  a  year 
in  the  required  field.  Four  of  the  five  years  were  above 
the  AF  yearly  total  average. 

The  next  command  examined  is  ESC  with  forty-nine 
computer  software  systems  with  a  low  total  per  line  average 
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cost  of  $.17.  Only  thirty-eight  of  the  systems  could  be 
analyzed  by  year  due  to  missing  data  in  the  year  field. 
ESC's  systems  are  relatively  new  suggesting  that  the  cost 
would  be  lower  than  earlier  systems  developed  elsewhere. 
This  appears  to  be  the  case.  Every  year  lists  a  lower 
yearly  average  per  line  cost  than  the  AF  total  average 
per  line  cost. 

MAC  had  the  third  most  systems  registered  with  153 
systems  averaging  $39.40  per  line  of  code.  This  is  almost 
eighteen  times  greater  than  the  AF  average  cost.  Only  five 
MAC  systems  lacked  a  year  identifier  with  the  remaining 
systems  covering  a  twenty-year  period.  Of  these  twenty 
years  analyzed,  thirteen  were  above  the  AF  yearly  total 
average  cost.  The  yearly  average  cost  per  line  was  skewed 
in  1967  due  to  a  large  maintenance  cost  on  one  system  that 
did  not  have  a  listing  for  the  number  of  lines  of  code.  As 
found  in  the  development  analysis,  the  other  system  for 
that  year  used  its  lines  of  code  figure  in  the  calculations 
for  both  systems.  If  the  first  system  is  removed  from  the 
calculations,  the  per  line  cost  is  reduced  to  $4.70.  The 
same  situation  occurs  again  in  1970.  By  deleting  the  com¬ 
puter  system  without  the  number  of  lines  of  code,  the  main¬ 
tenance  cost  per  line  is  reduced  from  $2107.43  to  $1120. 
Overall,  the  per  line  costs  are  well  above  the  range  of 
AF  costs  and  within  the  range  for  industry  averages. 
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PACAF  was  unusual  to  analyze  because  it  had  sixty- 
two  systems  registered  but  only  twenty- two  of  them  listed 
the  year  developed.  Of  these  twenty-two,  sixteen  were 
lacking  maintenance  cost.3  and  two  lacked  the  number  of 
lines  in  the  system.  With  the  remaining  information  for 
the  command,  the  total  average  maintenance  cost  per  line 
was  calculated  to  be  $.24.  This  is  below  the  AF  total 
average  cost  per  line.  Every  one  of  the  four  years  avail¬ 
able  for  comparison  was  below  the  AF  yearly  average  cost 
per  line.  All  four  were  below  three  dollars  per  line. 
However,  since  these  systems  are  relatively  new,  mainte¬ 
nance  costs  may  not  have  had  time  to  accumulate. 

SAC  was  examined  next.  The  130  systems  covered 
1963  through  1995,  although  twenty-two  systems  lacked 
appropriate  information  tc  categorize  by  year.  SAC's 
total  average  cost  per  line  is  $27.56.  This  is  nearly 
thirteen  times  greater  than  the  equivalent  AF  cost  per  line. 
For  the  twenty  years  examined,  SAC's  cost  per  line  was 
above  the  AF  average  yearly  cost  twelve  times.  Seven  years, 
SAC's  per  line  cost  were  in  the  same  range  as  industry’s. 
Surprisingly,  in  the  year  1982,  SAC  had  already  accumulated 
enough  maintenance  costs  to  boost  the  per  line  cost  to 
$716.57.  This  was  the  highest  maintenance  cost  in  1982 
of  any  of  the  commands. 

TAC  had  ninety-four  systems  listed  which  averaged 
a  per  line  cost  of  $6.14.  This  is  almost  three  times 
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greater  than  the  AF  average.  Only  seventy-one  of  these 
systems  listed  the  year  developed  covering  a  fourteen-year 
period.  Of  these  fourteen,  nine  of  the  years  were  less 
than  the  AF  yearly  average. 

USAFE  is  the  final  unique  command  examined.  It 
contained  sixty- three  records  which  gave  a  total  average 
cost  per  line  of  $1.36.  Only  fifty-six  records  had  the 
year  indicated.  These  records  covered  twelve  years.  The 
USAFE  yearly  total  cost  per  line  for  ten  years  was  below 
the  AF  yearly  total  cost  per  line. 

The  remaining  commands  were  grouped  together  under 
a  heading  of  "Other."  This  added  up  to  126  records 
covering  twenty-nine  commands.  Their  combined  records 
gave  a  total  average  per  line  cost  of  $15.76,  almost  seven 
times  larger  than  the  AF  total  average  cost  per  line.  The 
records  covered  a  total  of  twenty-one  years  from  1965  until 
1985.  During  this  period  the  "Other"  category  exceeded 
the  AF  yearly  average  cost  per  line  thirteen  times. 

Contractor  Maintenance  Costs.  Finally,  examining 
contractor  maintenance  costs  shows  a  total  average  main¬ 
tenance  cost  of  $12.70.  This  is  almost  six  times  greater 
than  the  AF  total  maintenance  cost  but  far  below  the  indus¬ 
try  average.  Most  of  these  systems  became  operational  in 
1981  or  later  which  may  explain  the  lesser  amount.  Main- 
tfc  ance  costs  have  not  been  able  to  accumulate  for  these 


58 


systems.  The  year-by-year  analysis  showed  that  contractor 
maintenance  costs  exceeded  AF  maintenance  cost  six  years, 
were  equivalent  three  years  and  below  the  remaining  six 
years.  These  were  below  the  industry  averages  in  all  but 
two  years,  1965  and  1966,  which  may  be  a  reflection  of  the 
extreme  age  of  the  systems  and  the  continued  accumulation 
of  maintenance  costs. 

Research  Objective  Four 

Have  these  maintenance  costs  shown  the  same 
increases  independent  of  the  AF  command  or  the  programming 
language  used? 

The  data  was  placed  on  graphs  and  examined  for  any 
patterns.  Initially  it  was  thought  that  maintenance  costs 
would  reflect  a  decrease  over  time;  that  is,  a  steadily 
decreasing  line  from  1955  to  1986.  The  logic  behind  this 
was  that  the  oldest  system  would  have  accumulated  the  most 
costs  in  the  maintenance  category.  This  is  not  the  case 
as  can  be  seen  in  the  total  AF  graph  in  Appendix  F.  The 
explanation  appears  to  be  that  the  oldest  systems  tend  to 
be  phased  out  leaving  only  a  few  systems  to  graph.  At 
the  same  time,  the  newer  systems  being  more  numerous  would 
show  a  greater  combined  maintenance  cost. 

Anouher  way  of  showing  this  would  be  to  take  the 
total  maintenance  cost  for  that  year  and  divide  that 
number  by  the  difference  between  1986  and  the  year 
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developed.  This  gives  the  average  amount  spent  each  year 
for  the  systems  developed  during  a  specific  year.  Perform¬ 
ing  this  on  every  year  in  the  total  AF  maintenance  cost 
table  shows  a  cyclical  but  steadily  rising  amount. 

This  method  also  shows  how  much  on  average  that 
the  AF  will  spend  this  year,  1986,  for  all  the  systems 
listed  in  AFR  700-19.  If  none  of  the  systems  is  removed 
from  inventory,  the  combined  amount  is  $236,339,850  spent 
on  maintenance  for  all  the  active  systems  listed. 

It  was  noted  in  this  analysis  that  maintenance 
costs  for  these  systems  generally  exceeded  the  development 
costs.  This  supports  the  industry  claim  that  maintenance 
costs  are  the  greater  part  of  the  total  life  cycle  costs. 
These  systems  are  still  operational  so  they  will  continue 
to  accumulate  maintenance  costs,  raising  the  percentage 
over  time.  Looking  at  the  year-by-year  total  AF  costs, 
twenty-one  of  the  thirty-one  years  examined  had  greater 
maintenance  costs  than  development  costs.  See  Appendix  F. 

Programming  Language.  Examining  the  data  further, 
the  costs  recorded  for  systems  developed  in  Assembler  showed 
total  development  costs  equivalent  to  total  maintenance 
costs.  The  year-by-year  analysis  showed  maintenance  above 
development  four  years,  below  six  years  and  equal  one  year. 
COBOL  showed  66  percent  of  the  total  life  cycle  costs  to 
be  maintenance.  The  year-by-year  analysis  showed 
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twenty-six  of  thirty-one  years  to  have  greater  maintenance 
costs  than  development  costs.  Systems  written  in  FORTRAN 
showed  greater  development  costs  than  maintenance  costs. 

Of  the  twenty-three  years  listed,  twelve  years  had  greater 
maintenance  costs.  These  statements  seem  contradictory; 
however,  closer  examination  shows  that  the  FORTRAN  systems 
had  several  extremely  large  development  costs.  This  skewed 
the  comparison,  overall,  but  not  in  the  year-by-year 
analysis.  The  "Other"  category  showed  development  costs 
greater  than  maintenance  costs  overall  and  for  eight  of  the 
fifteen  years  on  record.  See  Appendix  F. 

Command.  The  individual  ccmmands  showed  the  same 
situation.  AFAFC  had  greater  maintenance  costs  than 
development  costs  and  for  eight  of  the  twelve  years.  AFCC 
had  lower  total  maintenance  costs.  In  fourteen  of  the 
twenty-five  years  examined,  the  maintenance  costs  were 
equal  to  or  greater  than  development  costs.  AFLC  showed 
greater  total  maintenance  costs  and  for  yearly  totals  in 
twenty-one  of  the  twenty-seven  years  of  data.  AFSC  had 
greater  total  maintenance  costs  and  yearly  totals  for 
thirteen  of  twenty  years.  ATC  showed  almost  twice  the 
total  maintenance  costs  to  total  development  costs.  In 
thirteen  of  nineteen  years,  maintenance  was  either  equiva¬ 
lent  or  greater  than  development.  AU  showed  total  main¬ 
tenance  costs  to  be  65  percent  of  the  total  life  cycle 


costs.  Of  the  sixteen  years,  eleven  years  had  equivalent 
or  greater  maintenance  costs.  SISC,  with  only  a  small 
sample  to  use,  showed  maintenance  accounting  for  77  per¬ 
cent  of  the  total  life  cycle  costs.  Four  out  of  the  five 
years  also  showed  greater  maintenance  costs.  ESC  with  its 
relatively  new  systems  showed  greater  development  costs 
both  for  the  totals  and  in  three  out  of  six  years.  MAC 
followed  the  majority  of  the  commands  with  greater  total 
maintenance  costs  and  for  yearly  costs  for  seventeen  of 
twenty-one  years.  PACAF  had  much  lower  maintenance  to 
development  costs  for  both  the  totals  and  for  every  year. 
SAC's  data  shows  greater  maintenance  costs  for  the  totals 
and  for  eleven  of  twenty  years.  TAC  showed  greater  total 
development  costs.  Ten  of  the  fourteen  years  showed  a 
greater  development  costs  for  TAC.  The  final  command 
examined,  USAFE,  had  lower  total  maintenance  costs  overall 
and  for  every  year  examined.  The  "Other"  category  had 
greater  total  maintenance  costs  overall  and  for  fifteen 
of  twenty-one  years.  See  Appendix  F. 
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Research  Objective  Five 

What  differences  are  found  between  the  development 
and  maintenance  costs? 

Combining  the  information  of  research  objectives 
one  with  three  and  research  objectives  two  with  four 
showed  that  maintenance  costs  are  indeed  greater  than 
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development  costs  in  most  cases  despite  the  command  or 
programing  language.  See  Appendix  E  for  the  tables  and 
Appendix  F  for  the  graphs. 


advantages  over  AF  programmers.  They  have  developed  long¬ 
term  familiarity,  experience,  and  system  skills  that  are 
unavailable  to  AF  personnel.  Air  Force  personnel,  with 
three  to  four  years  per  duty  station,  do  not  have  adequate 
time  to  develop  those  skills  in  maintaining  computer  sys¬ 
tems  that  are  ten  to  thirty  years  old.  Therefore,  con¬ 
tractor  software  will  continue  to  be  an  important  factor 
in  future  software  maintenance. 

Research  Objective  One.  What  has  been  the  increase 
in  development  costs  over  time?  The  information  in  this 
study  has  not  shown  any  appreciable  rise  in  software  devel¬ 
opment  costs  over  the  last  thirty  years.  In  fact,  the  Air 
Force-wide  development  cost  per  line  in  1955  was  $20  per 
line.  The  same  cost  in  1985  was  only  $4.11.  The  costs 
during  this  thirty-year  period  fluctuated  but  the  last 
six  years  were  all  under  $10. 

Research  Objective  Two.  Have  these  development 
costs  shown  the  same  increase  independent  of  the  AF  command 
or  the  programming  language  used?  From  1955  to  1985,  the 
total  Air  Force  development  costs  rose  from  $2,661,500  to 

$63,037,789.  Over  this  thirty-year  period  the  costs  had  j 

1 

a  cyclical  but  steadily  rising  amount.  Assembler  and  j 

J 

FORTRAN  showed  a  decrease  in  use  while  COBOL  and  the  "Other"  J 

languages  have  shown  large  increases.  The  various  com-  j 

mands  listed  showed  cyclical  but  steady  increases  in  their  I 


software  development  costs  except  for  MAC,  PACAF,  SAC,  and 
TAC.  The  use  of  contractor  developed  software  also  showed  a 
steady  rise  from  $1,553,100  in  1955  to  $14,808,317  in  1984. 
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Research  Objective  Three.  What  have  been  the  costs 
of  software  maintenance  for  these  systems?  In  general,  the 
analysis  of  the  data  has  not  shown  any  rise  in  maintenance 
cost  per  line.  The  analysis  of  the  data  has  shown  a 
decrease  in  the  Air  Force  cost  per  line  from  $82.10  in  1955 
to  $5.45  in  1985.  Assembler,  COBOL,  and  FORTRAN  showed 
total  decreases  in  the  maintenance  cost,  although  these 
costs  showed  a  one  dollar  increase  from  $4.80  in  1964  to 
$5.80  in  1984.  Contractor  costs  went  down  from  $64  in  1965 
to  $17.44  in  1984.  All  of  these  costs  were  well  below  the 
industry's  average  of  $2,000  per  line  of  code. 


Research  Objective  Four .  Have  these  maintenance 
costs  shown  the  same  increase  independent  of  the  AF  command 
or  the  programming  language  used?  This  area  of  analysis 
showed  that  the  oldest  systems  have  accumulated  some  large 
maintenance  expenses  over  the  years.  The  rate  at  which 
these  costs  are  increasing  is  also  rising  over  time.  Where 
some  software  systems  required  thirty  years  to  accumulate 
maintenance  costs  equal  to  the  development  costs,  software 
systems  developed  as  late  as  1985  have  already  accumulated 
maintenance  costs  equal  to  or  greater  than  the  development 
costs.  The  accumulated  maintenance  costs  for  1955  were 
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$10,926,057  for  thirty  years  while  the  systems  developed 
in  1985  have  already  accumulated  $83 ,642, 709  in  maintenance 
costs.  All  four  language  categories  showed  cyclical  but 
steady  increases  in  the  rate  at  which  systems  accumulated 
maintenance  costs.  Also  shown  for  the  four  languages  was 
a  greater  rate  of  growth  in  maintenance  costs  for  the 
newer  systems  than  for  the  older  systems. 

An  analysis  of  maintenance  costs  of  the  four  cate¬ 
gories;  Air  Force,  languages,  commands,  and  contractor, 
revealed  two  things.  First,  commands  showed  strong 
cyclical  tendencies  but  revealed  no  significant  findings. 
Second,  maintenance  costs  on  contractor  developed  software 
increased  at  a  lower  rate  than  maintenance  costs  on  Air 
Force  developed  software.  The  money  spent  each  year  for 
software  maintenance  for  thirty  years  worth  of  software, 
averages  out  to  $23,339,850  per  year.  This  includes 
systems  operational  in  1986  that  fell  into  this  thirty- 
year  period. 

Research  Objective  Five .  What  differences  are 
found  between  the  development  and  maintenance  costs?  This 
thesis  has  shown  the  maintenance  cost  rising  at  a  greater 
rate  than  development.  Just  as  software  costs  in  industry, 
maintenance  costs  have  become  greater  than  development 
costs.  This  means  that  maintenance,  which  is  a  continual 
cost,  will  take  a  larger  and  larger  share  of  the 
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organization's  software  budget.  This  was  true  not  only 
for  the  different  languages  but  for  most  of  the  commands 
as  well,  except  for  ESC,  TAC,  and  USAFE. 

The  total  maintenance  costs  for  contractor  devel¬ 
oped  software  were  lower  than  the  development  costs.  This 
may  indicate  that  the  software  did  not  require  as  much 
maintenance  as  Air  Force  developed  software. 

Limitations  of  Research 

The  reader  should  understand  that  the  actual 
figures  given  in  this  database  for  development  and  main¬ 
tenance  costs  appear  very  low  compared  with  industry  costs. 
Verification  of  the  data  during  the  research  proved  impos¬ 
sible.  Without  this  verification,  several  questions  were 
raised.  First,  if  the  computer  system's  costs  show  figures 
similar  to  the  $20  and  $80  development  and  maintenance 
estimates  respectively,  does  this  mean  that  the  organiza¬ 
tion  did  not  record  their  costs?  AFR  700-19  stated  in  the 
directions  to  multiply  $20  by  the  number  of  lines  of  code 
in  the  system  to  give  the  estimated  development  cost. 

The  maintenance  cost  was  derived  by  multiplying  $80  by  the 
number  of  lines  in  the  system.  Second,  if  the  organization 
used  the  maintenance  cost  formula  from  AFR  700-19,  did  the 
organization  multiply  the  cost  by  the  total  lines  of  cede 
or  just  the  lines  of  code  changed? 
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Contractor  developed  software  showed  a  higher  cost 
per  line  than  Air  Force  developed.  Several  factors  could 
cause  this.  First,  contractors  are  driven  by  profits. 

They  must  charge  more  than  their  costs.  This  would 
usually  mean  that  contractor  developed  software  would  cost 
more  than  Air  Force  developed.  It  could  also  be  caused  by 
better  tracking  of  costs  on  the  part  of  the  contractors 
than  just  more  efficient  programming  on  the  part  of  the 
Air  Force.  Also,  the  fact  that  the  contractor  developed 
software  had  a  lower  percentage  of  maintenance  cost  to 
development  cost  than  the  Air  Force  may  indicate  more 
effective  software  development  or  better  maintainability 
built  into  this  software. 

"Off-the-shelf"  or  vendor  supplied  software  cost 
benefits  could  not  be  shown  due  to  an  inadequate  sample 
size . 

At  the  present  rate,  based  upon  this  study  and 
the  literature  search,  an  increase  in  the  software  budget 
and  the  number  of  programmers  will  be  required  to  maintain 
the  present  development  efforts.  This  is  due  to  the 
increasing  software  maintenance  requirements  placed  on 
organizational  resources. 

Recommendations 

Several  recommendations  are  being  made  as  a  result 
of  this  thesis.  First,  the  purpose  and  intentions  of 
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AFR  700-19  should  be  evaluated.  The  current  purpose  is  to 
serve  as  a- software  clearinghouse.  Nevertheless,  this 
database  is  the  only  database  in  the  DoD  and  A F  that  con¬ 
tains  software  ADPE  cost  data.  This  thesis  has  shown  that 
the  data  contained  in  the  ISD  database  does  not  accurately 
reflect  true  software  development  and  maintenance  costs. 

This  database  could  be  of  immeasurable  value  to  the 
AF  if  three  following  actions  were  implemented: 

1.  Modify  the  database  to  contain  fixed  length 
fields  with  well-defined  values  for  each  field.  This 
thesis  analyzed  only  a  small  subset  of  the  fields  available 
in  this  database.  AFR  700-19  contains  many  fields,  besides 
the  ones  used  in  this  thesis,  that  describe  the  hardware, 
software,  and  users  of  a  computer  system.  This  information 
can  provide  essential  but  currently  unavailable  cost 
analysis  in  many  areas  of  AF  computer  software.  Well 
defined  data  fields  would  help  ensure  validity  of  the  data. 

2.  AFR  700-19  should  be  modified  to  reflect  more 
rigorous  procedures  for  collecting  and  reporting  software 
development  and  maintenance  costs.  This  would  ensure  that 
the  data  submitted  by  the  major  commands  is  more  correct, 
complete,  and  consistent. 

3.  SISC  should  develop  a  set  of  statistical  analy¬ 
sis  to  track  software  cost  trends.  This  information  could 
then  be  used  by  the  major  commands  to  more  effectively 
allocate  software  resources. 
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Second,  it  is  recommended  that  commands  define  the 
responsibility  for  implementation  of  the  AFR  700-19  method¬ 
ology  to  collect  and  document  their  cost  data.  Referring 
back  to  Henderson  and  Sullivan,  "You  cannot  control  what 
you  do  not  measure  [emphasis  added] 11  (13:1)  .  The  costs 
derived  in  this  thesis  indicate  that  the  commands  do  not 
have  this  information  readily  available  and  indeed  they 
may  have  used  the  formula  provided  in  AFR  700-19  instead. 
These  methodologies  must  be  concise  because  programmers 
have  better  things  to  do  than  fill  out  accounting  forms. 
Automated  accounting  or  "tracking"  programs  are  becoming 
common  today  for  a  wide  variety  of  hardware  systems.  These 
programs  could  be  used  to  acquire  the  specific  information 
needed  for  AFR  700-19. 

Third,  it  is  recommended  that  in  the  interim,  until 
valid  cost  data  can  be  acquired,  commands  with  rising 
maintenance  costs  take  advantage  of  the  information  in 
Chapter  II  to  reduce  costs.  Appendices  G,  H,  and  I  provide 
a  more  in-depth  discussion  cf  these  concepts. 

Last,  it  is  recommended  that  continued  research 
be  performed  on  Air  Force  software  maintenance  to  answer 
two  questions: 

1.  What  benefits  occur  through  improvement  of 
maintainability  of  existing  software? 

2.  What  benefits  are  received  through  emphasizing 
maintainability  of  AF  software  during  software  development? 


Appendix  A:  Definition  of  Terms 


Sof twa re  Maintenance — the  performance  of  those  activities 
required  to  keep  a  software  system  operational  and  respon¬ 
sive  to  its  users  after  it  is  accepted  and  placed  into 
production.  Maintenance  does  not  include  coverting  soft¬ 
ware  from  one  machine  to  another  or  fran  one  programming 
language  to  another  (15:7).  Software  Maintenance  can  be 
broken  down  into  three  areas: 

a.  Perfective- -all  changes,  insertions,  deletions, 
modifications,  extensions,  and  enhancements  made  to  meet 
evolving  and  or  expanding  needs  of  user.  (Ex  -  making  code 
easier  to  understand,  improving  documentation,  optimizing 
the  code,  adding  a  new  capability.) 

b.  Adaptive — all  changes  which  are  initiated  as 

a  result  in  changes  in  the  environment.  (Ex  -  new  version 
of  an  operating  system,  new  peripheral  added  to  a  computer 
system,  new  version  of  a  compiler.) 


c.  Corrective — all  changes  necessitated  by  errors 
in  the  system.  (Ex  -  quick  fixes  and  firefighting,  aborts 
because  of  inability  to  handle  inputs.) 

There  is  a  great  deal  of  overlap  between  these  activities, 
and  the  skills  required  of  programmers  are  similar. 
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Appendix  B :  SAS  Program 


COPYRIGHT  (C)  1985  SAS  INSTITUTE  INC.,  CARY,  N.  C.  27511, 
U.S.A. 

NOTE:  VMS  VERSION  OF  SAS  RELEASE  5.03  AT  AIR  FORCE  INSTI¬ 
TUTE  OF  TECHNOLOGY  (03855004) . 

NOTE:  LICENSED  CPUID  MODEL  «  11/780,  SERIAL  *  01384A2A. 

DATA  THEFILE; 

INPUT  ISD  $  1-5  YEAR  6-7  CONTRACT  9  COMMAND  $  11-22 
LANG1  $  23-34 

PROG  35-40  LINES  41-50  DEVCOST  51-63  MANCOST  64-76; 

IF  CONTRACTS; 

IF  COMMAND* '  '  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND* 'AAC'  THEN  COMMAND*' OTHER* ; 

IF  COMMAND=  * AFAA *  THEN  COMMAND= ' OTHER' ; 

IF  COMMAND= 1 AFAF '  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND* *  AFCAC '  THEN  COMMAND*' OTHER' ; 

IF  COMMAND* ' AFCCMS '  THEN  COMMAND*' OTHER* ; 

IF  COMMAND* ' AFCCS '  THEN  COMMAND* 'OTHER* ; 

IF  COMMAND* ' AFDSDO*  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' AFESC*  THEN  COMMAND* ' ESC  * ; 

IF  COMMAND* '  APIS'  THEN  COMMAND*  '  OTHER*  ; 

IF  COMMAND*' AF ISC'  TKI  I  COMMAND*' OTHER' ; 

IF  COMMAND*' AFLCM'  THEN  COMMAND* 'AFLC ' ; 

IF  COMMAND*’ AFMEA*  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND* ' AFMFC '  THEN  C OMMAND* ' OTHER' ; 

IF  COMMAND* 'AFMSMET'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' AFMSC'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' AFOTEC'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' AFRES'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND** AFOTEC'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' AFRES'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND* 'AFWL'  THEN  COMMAND* ' OTHER' ; 

IF  C OMMAND* ' ANG  *  THEN  COMMAND* *  OTHER'  ; 

IF  COMMAND* 'ANG SC'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' DC A'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' DLA'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' DNA'  THEN  COMMAND*' OTHER' 

IF  COMMAND* ' JCS'  THEN  COMMAND*  *  OTHER 
IF  COMMAND*' JDA'  THEN  COMMAND* ' OTHER 
IF  COMMAND*' JDSSC '  THEN  COMMAND* ' OTHER  ; 

IF  COMMAND*' AFIT'  THEN  COMMAND* ' AU ' ; 

IF  COMMAND*' DFSEC '  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND*' DSDO'  THEN  COMMAND*' DSDO' ; 

IF  COMMAND* 'MMSSA'  THEN  COMMAND* 1  OTHER' ; 

IF  COMMAND* ; MED  SYS  DIV'  THEN  COMMAND* ' OTHER' ; 


IF  COMMAND*' SPACECMD'  THEN  COMMAND* ' OTHER 1 
IF  COMMAND* ' TACC '  THEN  COMMAND* ' TAC ' ; 

IF  COMMAND* ' TP SC'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND* 'USAF'  THEN  COMMAND* ' OTHER' ; 

IF  COMMAND* 'USAF A'  THEN  COMMAND* ' OTHER' ; 

IF  LANGl*' AFOLDS'  THEN  LANG 1= ' OTHER' ; 

IF  LANG1* ' ALGOL'  THEN  LANGl* ' OTHER' ; 

IF  LANG1* ' APL '  THEN  LANG1*’ OTHER' ; 

IF  LANG1* ' BASIC '  THEN  LANG1* ' OTHER' ; 

IF  LANG1*' BASIS'  THEN  LANGl* ' OTHER' ; 

IF  LANGl* ' DAD'  THEN  LANGl*' OTHER' ; 

IF  LANGl* 'DBMS'  THEN  LANGl* ' OTHER' ; 

IF  LANGl* 'GMAP'  THEN  LANGl* ' OTHER' ; 

IF  LANGl*' JOVIAL'  THEN  LANGl* ' OTHER' ; 

IF  LANGl* 'PASCAL'  THEN  LANGl* ' OTHER' ; 

IF  LANGl*' PL/1 '  THEN  LANGl* ' OTHER' ; 

IF  LANGl* ' RPG '  THEN  LANGl* ' OTHER' ; 

IF  LANGl*' RAMIS'  THEN  LANGl* ' OTHER’ ; 

IF  LANGl*' SIMSCRIPT'  THEN  LANGl* ' OTHER' ; 

IF  LANGl* 'UTILITY'  THEN  LANGl* ' OTHER' ; 

IF  LANGl*' UTILITIES'  THEN  LANGl*' OTHER' ; 

IF  LANGl*'  '  THEN  LANGl* ' OTHER’ ; 

CARDS; 

PROC ' SORT;  BY  YEAR; 

PROC  PRINT; 

BY  YEAR; 

VAR  LINES  DEVCOST  MANCOST; 

SUM  LINES  DEVCOST  MANCOST; 

FORMAT  LINES  DEVCOST  MANCOST; 

PROC  SORT;  BY  COMMAND  YEAR; 

PROC  PRINT; 

BY  COMMAND  YEAR; 

VAR  LINES  DEVCOST  MANCOST; 

SUM  LINES  DEVCOST  MANCOST; 

FORMAT  LINES  DEVCOST  MANCOST; 

PROC  SORT;  BY  LANGl  YEAR; 

PROC  PRINT; 

BY  LANGl  YEAR; 

VAR  LINES  DEVCOST  MANCOST; 

SUM  LINES  DEVCOST  MANCOST; 

FORMAT  LINES  DEVCOST  MANCOST; 


Appendix  C:  Cost  per  Line  Comparison  Graphs 

Appendix  C  contains  the  graphs  of  development  and 
maintenance  cost  per  line  over  time  for  total  AF,  the  four 
language  categories,  selected  AF  commands,  and  contractor 
developed  software. 

On  the  graph,  development  cost  is  represented  by 
a  solid  line.  Maintenance  cost  is  represented  by  a  dashed 
line.  This  legend  is  used  on  all  the  graphs  contained  in 
this  appendix. 
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Cost  per  Line  over  Time  Category  -  Command  -  AFAFC 
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Cost  per  Line  over  Time  Category  -  AF  Commands  -  ESC 
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Cost  per  Line  over  Time  Category  -  AF  Commands  -  Others 


Appendix  Dj  AFR  700-19  Database  Description 

I.  Information  System  Designator 

II.  System  Cede 

III.  Title 

A.  Acronym 

B.  Sensitivity 

C.  Criticality 

IV .  ADPE 

V.  Type  System 

VI.  Responsible  Offices 

A .  ADPS 

B.  ADS 

C.  Development  Center 

D.  Air  Staff  Functional  OPR 

E.  Collateral  Users 

VII.  Interface  with  other  Systems 

VIII.  Documentation  Reference 

A.  Computer  Operation  Manual  <OM) 

B.  Users  Manual  <UM) 

C.  Implementation  Date  of  System 

IX.  General 

A.  Progranming  Languages 

B .  Number  of  Programs 


C.  Lines  of  Code 

D.  Processing  Mode 

E.  Transition 

X.  Commercial  Software  Used 

A.  Vendor 

B.  Title 

C.  Acquisition  Basis 

D.  Cost 

XI.  ADS  Investment  Costs 

A.  Initial  Development  Cost 

B.  Maintenance  Cost 

C.  Total  Cost  thru  FY  84 

XII.  Authorizing  Directive 

XIII.  Functional  Description  of  System 

XIV.  Research  Results 
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Appendix  E:  Tables  of  Cost  Totals 


Appendix  E  contains  tables  developed  from  the 
data.  It  is  organized  according  to  category  and  year.  The 
number  of  lines  of  code  developed  during  that  year,  the 
total  development  cost,  the  development  cost  per  line  of 
code,  the  accumulated  maintenance  cost  for  the  systems 
developed  that  year,  and  the  maintenance  cost  per  line  of 
code.  This  information  is  organized  by  total  AF,  the  four 
language  categories,  selected  AF  commands,  and  contractor 
developed  software. 
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Tables  of  Cost  Totals  -  Languages  -  Assembler 
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Tables  of  Cost  Totals  -  Contractor 


Appendix  F  contains  the  charts  comparing  the  devel¬ 
opment  and  maintenance  cost  per  year  over  time  for  total 
AF,  the  four  language  categories,  selected  AF  commands,  and 
contractor  developed  software. 

On  the  chart,  development  cost  is  represented  by  a 
solid  bar.  Maintenance  cost  is  represented  by  a  diagonally 
slashed  bar.  This  legend  is  used  on  all  the  charts  con¬ 
tained  in  this  appendix. 


Total  Software  Costs  -•  Languages  -  Assembler 


Total  Software  Costs  -  Languages  -  COBOL 


f  Total  Software  Costs  -  Languages  -  FORTRAN 


Graphs  of  Total  Software  Costs  -  Command  -  AFAFC 


Graphs  of  Total  Software  Costs  -  Command  -  AFCC 


20,000,000 


Graphs  of  Total  Software  Costs  -  AF  Commands 


Total  Software  Costs  -  AF  Commands 


Graphs  of  Total  Software  Costs  -  AF  Commands 


Graphs  of  Total  Software  Costs  -  AF  Commands  -  ESC 
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Graphs  of  Total  Software  Costs  -  AF  Commands 


35.000 


Total  Software  Costs  -  AF  Commands  -  PACAF 
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tware  Costs  -  AF  Commands 


Software  Costs  -  AF  Commands 


Graphs  of  Total  Software  Costs  -  AF  Commands  -  TAC 


Total  Software  Costs  -  AF  Commands  -  USAFE 


Graphs  of  Total  Software  Costs  -  AF  Commands  -  Others 
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Appendix  G:  Software  Quality  Assurance 

Factor  Tradeoffs 


Factor 

Definition 

Correctness 

Extent  to  which  a  program  satisfies  its 
specifications  and  fulfills  the  user's 
mission  objectives. 

Reliability 

Extent  to  which  a  program  can  be  expected 
to  perform  its  intended  function  with 
required  precision. 

Efficiency 

The  amount  of  computing  resources  and 
code  required  by  a  program  to  perform  a 
function. 

Integrity 

Extent  to  which  access  to  software  or 
data  by  unauthorized  persons  can  be  con¬ 
trolled. 

Usability 

Effort  required  to  learn,  operate,  pre¬ 
pare  input,  and  interpret  output  of  a 
program. 

Maintainability 

Effort  required  to  locate  and  fix  an 
error  in  an  operational  program. 

Testability 

Effort  required  to  test  a  program  to 
ensure  it  performs  its  intended  function. 

Flexibility 

Effort  required  to  modify  an  operational 
program. 

Portability 

Effort  required  to  transfer  a  program 
from  one  hardware  configuration  and/or 
software  system  environment  to  another. 

Reusability 

Extent  to  which  a  program  can  be  used  in 
other  applications — related  to  the 
packaging  and  scope  of  the  functions  that 
the  programs  perform. 

Interoperability 

Effort  required  to  couple  one  system 
with  another  (21:22). 

Criterion 

Definition 

ri  elated 

Factors 

Traceability 

Those  attributes  of  the  software  that  pro* 
vids  a  thread  from  tha  requirements  to  the 
implementation  with  respect  to  the  specific 
development  and  operational  environment. 

Correctness 

Completeness 

Those  attributes  of  the  software  that  pro¬ 
vide  full  implementation  of  the  functions 
required. 

Correctness 

Consistency 

Those  attributes  of  tha  software  that  pro¬ 
vide  uniform  design  and  implementation 
techniques  and  notation. 

Correctness 

Reliability 

Maintainability 

Accuracy 

Those  attributes  of  the  software  that  pro¬ 
vide  the  required  precision  in  calculations 
and  outputs. 

Reliability 

Error  Toi crane  .* 

Those  attributes  of  the  software  that  „-ro- 
vida  continuity  of  operation  undt  :  non- 
nominal  conditions. 

Reliability 

Simplicicy 

Those  artriDutes  of  the  software  th  .  pro¬ 
vide  implementation  of  functions  ir  me 
most  understandaale  manner.  Us  ally 
avoidance  of  practices  which  me  .sse  cam- 
piemty.l 

Reliability 

Maintainability 

Testabii  y 

Modularity 

Those  attributes  of  the  software  that  pro- 
•  vide  a  structure  of  highly  independent 
modules. 

Maintainability 

Flexibility 

Testability 

Portroility 

Reusability 

Interoperability 

Generality 

Those  attributes  of  the  software  that  pro- 

Flexibility 

vide  breadth  to  the  functions  performed. 

Reusability 

Expandability 

Those  attnbutes  of  the  software  that  pro¬ 
vide  for  expansion  of  data  storage  re¬ 
quirements  or  computational  functions. 

Flexibility 

Instrumentation 

Those  attributes  of  the  software  mat  pro¬ 
vide  for  the  measurements  of  usage  or 
identification  of  errors. 

Testability 

Self- 

Those  attributes  of  the  software  that  pro- 

Flexibility 

Descriptiveness 

vide  explanation  of  the  implementation 
of  a  function. 

Maintainability 

Testability 

Portability 

Reusability 

(21:25) 

Criterion 

Definition 

Related 

Factors 

Execution 

Efficiency 

Those  attributes  of  the  software  that  pro¬ 
vide  for  minimum  processing  time. 

Efficiency 

Storage  Efficiency 

Those  attributes  of  the  software  that  pro¬ 
vide  for  minimum  storage  requirements 
during  operation. 

Efficiency 

Access  Control 

Those  attributes  of  the  software  that  pro¬ 
vide  for  control  of  the  access  of  software 
and  data. 

Integrity 

Access  Audit 

Those  attributes  of  the  software  that  pro¬ 
vide  for  an  audit  of  the  access  of  software 
and  data. 

Integrity 

Operabiity 

Those  attributes  of  the  software  that  deter¬ 
mine  ope'ation  and  procedures  concerned 
with  the  operation  of  the  software. 

Usability 

Training 

Those  attributes  of  the  software  that  pro¬ 
vide  transition  from  current  operation  or 
initial  familiarization. 

Usability 

Communicativeness 

Those  attributes  of  tne  software  mat  pro¬ 
vide  useful  inputs  and  outputs  wsicn  can 

be  assimilated. 

Usability 

Software  System 

Those  attributes  of  the  software  that  deter¬ 

Portability 

Independence 

mine  its  dependency  on  tne  software  en¬ 
vironment  (operating  systems,  utilities, 
input/ output  routines,  etc.). 

Reusability 

Machine 

Those  attributes  of  the  software  that  deter¬ 

Portability 

Independence 

mine  its  dependency  on  the  hardware 
system. 

Reusability 

Communications 

Commonality 

Those  attributes  of  the  software  that  pro¬ 
vide  the  usa  of  standard  protocols  and 
interface  routines. 

Interoperability 

Data  Commonality 

Those  attributes  of  the  software  that  oro- 
vide  the  usa  of  standard  data  representa¬ 
tions. 

Interoperability 

Conciseness 

Those  attributes  of  the  software  that  pro¬ 
vide  for  implementation  of  a  function  with 
a  minimum  amount  of  code. 

Maintainability 

(21:26) 


Appendix  H:  Definitions  of  Maintenance  Activities 


Requirements  Analysis 

Evaluation  of  the  impact  of  a  change  in  requirements  or 
the  reason  why  a  current  requirement  is  not  satisfied. 
Determination  and  assimilation  of  the  current  software 
documentation  necessary  to  understand  the  nature  of 
the  change  required. 

Design  Analysis 

Evaluation  of  the  impact  of  a  modification  and  develop¬ 
ment  of  a  strategy  of  redesign.  A  decision  typical  of 
this  activity  is  whether  a  portion  of  the  system  has  to 
be  redesigned  or  whether  the  modification  can  be  made 
within  the  context  of  the  current  software  architecture. 
Also  included  is  an  evaluation  of  modification  to  the 
data  base  design. 

User  Interface 


Activities  associated  with  interacting  with  the  user/ 
customer.  This  activity  may  involve  formal  documenta¬ 
tion,  meetings,  and  walkthroughs,  etc. 

Design  Review 

Formal  or  informal  review  of  the  design  analysis 
activity. 

Problem  Report  Recording  and  Control 

Includes  all  activities  associated  with  how  users 
report  problems,  how  problems  are  logged,  assigned 
priority,  response  time,  and  closed. 

Configuration  Control 

Includes  all  activities  associated  with  maintaining 
baseline  version  of  code. 

Data  Base  Modification 


Modification  made  to  data  base  structure  and  individual 
data  values. 
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Code  Modif  ication/Recoropilation 


Changes  made  to  code  by  programmers  to  repair  an  error 
or  to  enhance  the  operation  of  the  system. 

Code  Debugging/Module  Testing 

Includes  testing  after  code  changes  have  been  made  and 
investigative  debugging,  i.e.  testing  to  identify  where 
an  error  source  is. 

Subsystem  Testing 

Testing  groups  of  modules  or  programs  to  assess  whether 
modifications  have  been  made  correctly. 

System  Testing 

Tests  run  to  determine  if  new  version  of  software,  due 
to  corrective,  perfective,  or  adaptive  maintenance, 
operates  correctly.  Typically  called  regression  test¬ 
ing  if  same  set  of  test  cases/ test  data  used.  Accept¬ 
able  completion  of  these  tests  is  usually  the  basis 
for  fielding  the  new  version  of  the  system. 

Documentation  Modification 


Activities  including  changes  to  system  specifications, 
users  manuals,  maintenance  manuals,  etc.,  made  as  a 
result  of  modification  to  system. 

Standards  Audit 


Activities  performed  to  insure  new  version  of  system 
and  documentation  meet  established  standards  prior  to 
fielding. 

Code  Inspections/Walkthroughs 

Review  of  modifications  to  code. 

Test  Data  Generation 


Development  of  test  data  to  verify  and  validate  code 
changes. 

Management  Planning  and  Control 

Management  activities  related  to  planning,  control,  per¬ 
sonnel  assignment,  prioritization  of  jobs,  personnel 
requirements  estimation,  budget  estimation  and  control, 
scheduling,  etc. 
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Field  Delivery 


Activities  associated  with  fielding  updated  system. 

Software  Support  Development/Maintenance 

Development  and  maintenance  of  tools  used  in  support¬ 
ing  above  activities. 

Hardware  Support 

Procurement  and  maintenance  of  hardware  system  used  as 
maintenance  facility. 

Administrative  Support 

Secretarial,  data  entry,  and  clerk  support  to  main¬ 
tenance  personnel  (19:9-12). 
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Appendix  I :  Metrics  of  Maintenance 


Basic  Measure  of  Maintenance  Activities: 

1.  Number  of  Lines  of  Code/Number  Modified. 

2.  Number  of  Data  Items/Number  Modified. 

3.  Number  of  Modules /Number  Modified/Number  Added. 

4.  Number  of  Func tions/'Number  Modified/Number  Added. 

5.  Number  of  Interfaces/Number  Modified/Number  Added. 

6.  Number  of  Requirements/Structure  Changes/Number 

Changed/Number  Added/Links  Changed. 


Measures  of  Maintainability  of  System: 

1.  Traceability — Measure  of  traceability  from  require¬ 

ments  to  code. 

2.  Consistency — Measures  of  use  of  standard  data 

definition,  naming,  documentation  conventions 
and  Requirements  Consistency. 

3.  Conciseness — Halstead's  Length/Effort  Measures. 

4.  Modularity — Lines  of  code  by  module  profile  Called/ 

call  matrix. 

5.  Self-description — Number  of  comments/ LOG. 

6.  Stability — Myer's  Stability  Measure. 

7.  Effort/Cost — CPU  run  time.  Average  time  to  fix. 

8.  Complexity — McCabe's  Cyclomatic  Number. 


Measures  of  Reliability  of  System: 

1.  Number  of  User  Problem  Reports. 

2.  Number  of  User  Problem  Reports  per  line  of  code. 

3.  Number  of  User  Problem  Reports  induced  as  a  result 

of  a  maintenance  activity. 

4.  Number  of  User  Problem  Reports  induced  as  a  result 

of  a  maintenance  activity  per  line  of  code 
modified. 

5.  System  Reliability  { MTBF ) . 

6.  System  Availability  .MTBF  -  MTTR.  . 

'  MTBF 

7.  Completeness  of  Requirements. 

8.  Error  Categorization/ Impact  Assessment  {19; 70). 
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